@drkmttr/adhere 0.7.3 → 0.8.0

package/README.md

@@ -8,9 +8,10 @@ model) whether the file breaks each rule, gets a calibrated probability per
 rule, and reports the ones above a threshold with the line Jev points at. The
 report uses the same frame as `vp lint`.
 
-Deterministic rules (substring matches, type checks) belong in a normal linter.
+Deterministic rules (substring matches, type checks) belong in a normal linter;
+`adhere validate` points out a rule that looks like one.
 
-A preset for effect is included which can be run without setup: 
+A preset for effect is included which can be run without setup:
 
 ```sh
 TYPESAFE_API_KEY=xxx npx @drkmttr/adhere lint --preset effect
@@ -36,20 +37,25 @@ with a TypeSafe AI API key: the one `adhere login` saved, or
 
 ```sh
 adhere init                          # .adhere/config.ts and two example rules
-echo ".adhere/cache/" >> .gitignore
 adhere login                         # save your TypeSafe AI API key, once
-adhere validate                      # Checks your rules for contradictions using Jev
+adhere validate                      # Checks your rules' wording, and contradictions using Jev
 adhere lint                          # audit the working directory
+git add .adhere                      # commit the rules, and the judgments they cost
 ```
 
 Without a config or rules, `adhere lint --preset effect` audits against the
 built-in Effect rules. A finding looks like this:
 
 ```text
-  × data/brand-ports (0.93): A port is a branded, range-checked integer, not a bare number.
-   ╭─[src/server.ts:4:1]
- 4 │ const port: number = Number(process.env.PORT ?? 3000);
-   · ──────────────────────────────────────────────────────
+  × data/brand-ports (0.93): A port must be a branded, range-checked integer, never a bare number.
+   ╭─[src/server.ts:6:3]
+ 4 │ export const serve = Effect.gen(function* () {
+ 5 │   const host = process.env.HOST ?? "localhost";
+ 6 │   const port: number = Number(process.env.PORT ?? 3000);
+   ·   ──────────────────────────────────────────────────────
+ 7 │   yield* listen({ host, port });
+ 8 │   yield* Effect.log(`Listening on ${host}:${port}`);
+ 9 │ });
    ╰────
   hint: const Port = Schema.Int.pipe(
           Schema.check(Schema.isBetween({ minimum: 1, maximum: 65535 })),
@@ -60,7 +66,10 @@ Found 1 error.
 42 files, 3 judged, 39 cached.
 ```
 
-The header is the rule id, Jev's probability, and the rule's description. The
+The header is the rule id, Jev's probability, and the rule's description.
+Under it is the line Jev points at, underlined, with the code around it: the
+largest statement around the line that is 30 lines or fewer, usually the whole
+function, and up to 3 lines of whole statements on either side. The
 hint is the rule's code that must be written; a rule with only code that must
 never be written shows that code, labeled `never:`, instead. On a terminal, the report is in color and the code
 in it is highlighted. The exit code is 0 when nothing is reported and 1 when
@@ -78,7 +87,8 @@ adhere lint --yes              # send the requests without asking first
 adhere lint --limit 500        # judge at most 500 checks; the rest wait for the next run
 adhere lint --rpm 30           # send at most 30 requests a minute
 adhere lint --filter 'src/**'  # read only the files a glob matches
-adhere validate                # load the config and rules, ask Jev whether any contradict
+adhere lint --log-level debug  # log each request to Jev on stderr
+adhere validate                # check the rules' wording; ask Jev whether any contradict
 adhere init [--force]          # scaffold .adhere/config.ts and two example rules
 adhere login                   # save a TypeSafe AI API key for later runs
 adhere logout                  # delete the saved key
@@ -129,6 +139,22 @@ more, and start a pattern with `!` to leave out what it matches. Together:
 adhere lint --filter 'packages/api/**' --filter '!**/generated/**' --limit 200 --rpm 30
 ```
 
+### Logging a run
+
+`--log-level debug` logs a run's work on stderr: the config it loaded, how many
+paths it listed and how many of them it reads, and each request to Jev, with
+the file it is for, about how many tokens it carries, the HTTP status, how long
+it took, and Cloudflare's Ray ID. A failed attempt is logged even when a retry
+hides it. `--log-level trace` adds each file as it is read, planned, and
+answered from the cache, and the first 4000 characters of any error Jev's API
+answers with. The counter stays off at these levels, since its redraws would
+garble the lines, and stdout still carries only the report, so the log can go
+to a file of its own:
+
+```sh
+adhere lint --log-level debug 2> adhere.log
+```
+
 ### What gets read
 
 `adhere lint` reads the `.ts` files under the working directory, except `.d.ts`,
@@ -150,16 +176,45 @@ kept, and no second one is added, even with `--force`.
 
 `adhere validate` loads the config, rule files, and presets the way `lint`
 does, so a file that fails to decode refuses here too, and it prints how many
-rules it loaded. `--preset` adds a built-in rule set, as for `lint`. It then
-asks Jev whether any two rules that apply to the same files contradict, so that
-no code can follow both: one request names, per rule, the rule it conflicts
-with, if any, and each named pair then gets a probability from a request of
-its own, holding only those two rules, since other rules beside them dilute
-the judgment. A pair above
-the threshold is printed with both rule files, and the exit code is 1. Two
-rules with the same id are not compared: a nested rule that shares an id
-shadows the other on purpose. When no two rules share files nothing is sent;
-otherwise `validate` needs the API key, as `lint` does.
+rules it loaded. `--preset` adds a built-in rule set, as for `lint`.
+
+Next it checks each rule's wording against the
+[rule writing tips](#rule-writing-tips), which sends nothing, and prints each
+rule worded otherwise with its file and how:
+
+- its description does not say a word its code is under, such as "never"
+  beside a `never` block;
+- it has code to write but no example of code that breaks it;
+- its description says the other kind of rule's words: "should" in a
+  requirement, or "must" or "never" in a guideline.
+
+A rule with only code that must never be written needs no `must` example, as
+[Rules as Markdown files](#rules-as-markdown-files) explains. The tips are
+advice, so the wording does not change the exit code.
+
+Then it reports the linter check, which also sends nothing. While `lint`
+judges a file against one of the project's own rules, it asks Jev a second
+question beside it: should the rule have been checked by a regular linter, that
+is, could a linter or type checker have decided exactly whether this file
+follows it? It asks on 10 files per rule, spread over the files it judges, and
+keeps the answers in the cache until the rule's text changes. `validate` prints
+each rule flagged on 7 or more of its 10 files as one a regular linter should
+probably check, and each flagged on 3 to 6 as a rule that reads differently
+from file to file, whose description should say more precisely what it
+applies to. A rule with fewer than 10 answers is counted as waiting; `lint`
+collects them as it judges files, so a repo whose files are all cached collects
+them as its files change. Preset rules are left out: they are not the
+project's to change. The linter check is advice too.
+
+It then asks Jev whether any two rules that apply to the same files
+contradict, so that no code can follow both: one request names, per rule, the
+rule it conflicts with, if any, and each named pair then gets a probability
+from a request of its own, holding only those two rules, since other rules
+beside them dilute the judgment. A pair above the threshold is printed with
+both rule files, and the exit code is 1. Two rules with the same id are not
+compared: a nested rule that shares an id shadows the other on purpose. When
+no two rules share files nothing is sent; otherwise `validate` needs the API
+key, as `lint` does.
 
 ### Login
 
@@ -309,18 +364,47 @@ violations with the fewest false positives. In general:
 
 On the Effect preset, rules written this way cut the findings Jev got wrong at
 the default threshold from 6 to 1, and caught as many violations. For more
-about the evaluations, see [the eval](eval/README.md).
+about the evaluations, see [the eval](eval/README.md). `adhere validate` lists
+each rule worded otherwise.
 
 ### Presets
 
 A preset has the shape of a config without `presets`: `rules`, and optionally
-`model` and `threshold`. `effect` is the one preset: 26 Markdown rules in
+`model` and `threshold`. `effect` is the one preset: 19 Markdown rules in
 [`presets/effect/`](./presets/effect/), lifted from the
 [effect-solutions](https://github.com/kitlangton/effect-solutions) docs and the
 [effect/platform](https://effect.website/docs/platform/introduction/) docs.
 Name it in the config, or on the command line with `--preset effect`, in which
 case the config file is optional.
 
+The preset leaves out conventions a linter checks exactly. Effect's language
+service, [`@effect/tsgo`](https://github.com/Effect-TS/tsgo) on TypeScript 7,
+checks these seven, which the preset checked through 0.7. Most are off by
+default; `npx @effect/tsgo setup` installs it, and these lines in its plugin
+options in `tsconfig.json` turn them on:
+
+```jsonc
+"diagnosticSeverity": {
+  "strictEffectProvide": "warning", // layers are provided once, at the entry point
+  "leakingRequirements": "warning", // service methods have no requirements
+  "effectFnOpportunity": "warning", // a named function returning an Effect uses Effect.fn
+  "preferSchemaOverJson": "warning", // JSON is decoded with Schema, not JSON.parse
+  "nodeBuiltinImport": "warning", // platform services, not node: builtins
+  "globalFetch": "warning",
+  "globalFetchInEffect": "warning",
+  "processEnv": "warning",
+  "processEnvInEffect": "warning",
+  "globalRandom": "warning", // the Random service, not Math.random
+  "globalRandomInEffect": "warning",
+  "globalErrorInEffectFailure": "warning", // tagged errors, not Error
+  "extendsNativeError": "warning"
+}
+```
+
+`npx @effect/tsgo diagnostics --project tsconfig.json` runs them in CI. Bun
+globals, which the old platform rule also ruled out, need a lint rule of their
+own, such as `no-restricted-globals`.
+
 ### Precedence
 
 Highest first: `--threshold` on the command line, the config file, presets in
@@ -340,7 +424,9 @@ beats all of the above for that rule.
    at the rule's scope, so a file with no code the rule is about is a no. Since
    no rule sits in the shared state, a rule's probability depends only on the
    file and that rule, not on which other rules share the request. Every
-   question shares the state cost of the request.
+   question shares the state cost of the request. On up to 10 files per
+   project rule, the rule's question has a second one beside it, the linter
+   check (see [Validate](#validate)), with the same fields.
 2. A second request only when at least one rule's probability is above its
    threshold: one `choice` question per flagged rule over the file's non-blank
    lines, which yields the line to report. A file with more than 255 non-blank
@@ -356,15 +442,42 @@ beats all of the above for that rule.
 
 ## Cache
 
-Judgments are cached in `.adhere/cache/` (add it to `.gitignore`), one entry
-per file. An entry stores the file's content hash and, per rule, the
-probability, the located line, and a fingerprint of the model and the
-question asked for the rule, which carries the rule's text. A changed file
-re-judges every rule for that file. An edited rule re-judges only that rule,
-and an adhere that asks its questions differently re-judges every rule once.
-A lowered threshold locates cached judgments that are newly above it without
-judging again. Entries depend on content, not on the
-machine, so restoring `.adhere/cache/` between CI runs skips unchanged files.
+Judgments are cached in `.adhere/cache/`. Commit it. Every judgment is a paid
+request, and Jev's answers vary a little from run to run, so a committed cache
+gives everyone and CI the same findings without paying for them again, and a
+pull request changes the judgments of only the files it changes.
+
+`files/` holds Jev's answers by the content they are about: each cache file is
+named for the hash of a source file's content, so a moved or copied file keeps
+its judgments, and identical files share them. Each answer sits under a
+fingerprint of the model and the question asked for the rule, which carries
+the rule's text. A changed file re-judges every rule for that file. An edited
+rule re-judges only that rule, and an adhere that asks its questions
+differently re-judges every rule once. A lowered threshold locates cached
+judgments that are newly above it without judging again. `tallies/` keeps the
+linter check's answers, one tally per rule, under a key that changes with the
+rule's text and the model.
+
+A cache file is written once and never changed: it is also named for the hash
+of its own text. Two branches that judge the same code add files rather than
+edit them, so git merges the cache without a conflict. After a run that reads
+every file, adhere prunes the cache: it deletes answers about content no file
+has and answers to rule texts no rule asks, and folds what is left into one
+file per content. A run narrowed by `--filter` does not prune. To keep the
+cache out of diffs, mark it as generated in `.gitattributes`:
+
+```text
+.adhere/cache/** linguist-generated -diff
+```
+
+Anyone who can commit can also write a cache file saying that code passes.
+Where lint gates a merge, lint a pull request against the cache as merged,
+not as the pull request has it; only the files it changes are judged again:
+
+```sh
+rm -rf .adhere/cache && git checkout origin/main -- .adhere/cache
+adhere lint --yes
+```
 
 The API key is read only when a request is about to be sent. A run where
 every file is cached needs no key and no network.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@drkmttr/adhere",
-  "version": "0.7.3",
+  "version": "0.8.0",
   "private": false,
   "description": "Lints a repository against the rules in .adhere/, judged by Jev.",
   "license": "MIT",
@@ -11,6 +11,12 @@
   "bin": {
     "adhere": "bin/adhere.js"
   },
+  "files": [
+    "bin",
+    "src",
+    "presets",
+    "skills"
+  ],
   "type": "module",
   "imports": {
     "#*": "./src/*"
@@ -18,12 +24,6 @@
   "exports": {
     ".": "./src/index.ts"
   },
-  "files": [
-    "bin",
-    "src",
-    "presets",
-    "skills"
-  ],
   "publishConfig": {
     "access": "public"
   },
@@ -68,10 +68,10 @@
     }
   },
   "optionalDependencies": {
-    "@drkmttr/adhere-darwin-arm64": "0.7.3",
-    "@drkmttr/adhere-darwin-x64": "0.7.3",
-    "@drkmttr/adhere-linux-arm64": "0.7.3",
-    "@drkmttr/adhere-linux-x64": "0.7.3",
-    "@drkmttr/adhere-win32-x64": "0.7.3"
+    "@drkmttr/adhere-darwin-arm64": "0.8.0",
+    "@drkmttr/adhere-darwin-x64": "0.8.0",
+    "@drkmttr/adhere-linux-arm64": "0.8.0",
+    "@drkmttr/adhere-linux-x64": "0.8.0",
+    "@drkmttr/adhere-win32-x64": "0.8.0"
   }
 }

package/presets/effect/basics/external-calls-are-resilient.md

@@ -1,5 +1,5 @@
 ---
-description: A call over the network, such as an HTTP request, a database query, or a third-party API, must carry a timeout and a retry schedule. Calls through the platform FileSystem and Path services are local and are not in scope.
+description: A call over the network, such as an HTTP request, a database query, or a third-party API, must be made with a timeout and a retry schedule, never bare. Calls through the platform FileSystem and Path services are local and are not in scope.
 ---
 
 ```ts must

package/presets/effect/testing/test-clock-for-time.md

@@ -1,5 +1,5 @@
 ---
-description: A test that depends on time must use TestClock. Real sleeps and it.live must be used only when real time is required.
+description: A test that depends on time must use TestClock, never a real sleep or it.live, unless the test needs real time.
 ---
 
 ```ts must

package/skills/adhere/SKILL.md

@@ -103,14 +103,15 @@ export default {
 } satisfies Config;
 ```
 
-Add `.adhere/cache/` to `.gitignore`. Uncached files need a TypeSafe AI API
+Commit `.adhere/cache/` with the code, so the team and CI reuse its judgments
+instead of paying for them again. Uncached files need a TypeSafe AI API
 key: one saved by `adhere login`, or `TYPESAFE_API_KEY`, which takes
 precedence; in this org it comes from SOPS via `just shell`.
 
 ## 5. Run and calibrate
 
 ```sh
-adhere validate              # decodes the config and rules, asks Jev about contradictions
+adhere validate              # decodes the config and rules; reports their wording, rules a linter could check, and contradictions
 adhere lint                  # or: adhere lint --preset effect
 adhere lint --threshold 0.9  # re-reads the cache, sends nothing
 ```

package/src/cli.ts

@@ -8,6 +8,7 @@ import {
   executeAudit,
   type FileDone,
   planAudit,
+  pruneCache,
   render,
 } from "#workflows/audit.ts";
 import {
@@ -20,16 +21,20 @@ import {
 import type { Overrides } from "#config.ts";
 import { findContradictions, formatContradictions } from "#contradictions.ts";
 import { initProject } from "#init.ts";
+import { formatLinterCheck, tallyProjectRules } from "#mechanical.ts";
 import { type PresetName, presetNames } from "#presets.ts";
 import { globalRuleSet } from "#rules.ts";
+import { formatStrays, straysAmong } from "#wording.ts";
 import {
   Console,
   Effect,
   Exit,
   Layer,
+  LogLevel,
   Option,
   Path,
   Redacted,
+  References,
   Runtime,
   Schema,
   Stdio,
@@ -118,8 +123,9 @@ const toStderr = (text: string) =>
  * anything else it stays quiet, so a log gets the plan and the report rather
  * than a line per file. A failed run keeps its last count on screen.
  */
-const counterFor = (plan: AuditPlan) => {
-  const live = process.stderr.isTTY === true;
+const counterFor = (plan: AuditPlan, logging: LogLevel.LogLevel) => {
+  // Log lines at debug or below would land between redraws and garble the counter.
+  const live = process.stderr.isTTY === true && LogLevel.isGreaterThan(logging, "Debug");
   let progress: Progress = { files: 0, requests: 0, findings: 0 };
   const draw = () => (live ? toStderr(`\r\u001b[2K${progressLine(progress, plan)}`) : Effect.void);
   return {
@@ -170,11 +176,13 @@ export const lintCommand = Command.make(
           return yield* Cancelled.make({});
         }
       }
-      const counter = counterFor(plan);
+      const counter = counterFor(plan, yield* References.MinimumLogLevel);
       yield* counter.start;
       const result = yield* executeAudit(plan, counter.update).pipe(
         Effect.onExit((exit) => counter.finish(Exit.isFailure(exit))),
       );
+      // Only a run that read every file knows what the cache still needs.
+      if (input.filter.length === 0) yield* pruneCache(plan);
       const color = yield* stdio.stdoutIsTerminal;
       const path = yield* Path.Path;
       // One write: separate Console.log calls have interleaved out of order here.
@@ -201,31 +209,44 @@ export const lintCommand = Command.make(
 
 /** Jev, with the key `lint` uses, and the config it reads the model and threshold from. */
 export const validateLayer = (overrides: Overrides) =>
-  JevLive.pipe(
-    Layer.provide([FetchHttpClient.layer, CredentialsLive]),
-    Layer.provideMerge(AdhereConfigLive(overrides)),
-  );
+  Layer.merge(
+    JevLive.pipe(Layer.provide([FetchHttpClient.layer, CredentialsLive])),
+    AuditCacheLive,
+  ).pipe(Layer.provideMerge(AdhereConfigLive(overrides)));
 
 /**
- * `adhere validate`: everything `lint` loads, then Jev on whether any two rules
- * that apply to the same files contradict. A config, rule file, or preset that
- * does not decode refuses here the way it would refuse `lint`.
+ * `adhere validate`: everything `lint` loads, then how each rule's wording
+ * differs from the README's rule writing tips, then the linter check's tallies
+ * from lint, then Jev on whether any two rules that apply to the same files
+ * contradict. Only a contradiction fails the run: the tips and the linter
+ * check are advice. A config, rule file, or preset that does not decode
+ * refuses here the way it would refuse `lint`.
  */
 export const validateCommand = Command.make("validate", { preset }, () =>
   Effect.gen(function* () {
     const config = yield* AdhereConfig;
     const path = yield* Path.Path;
-    const entries = config.scopedRules ?? globalRuleSet(config.rules, path.resolve());
-    const contradictions = yield* findContradictions(entries, config.threshold);
+    const root = path.resolve();
+    const entries = config.scopedRules ?? globalRuleSet(config.rules, root);
     const loaded = `${entries.length} ${entries.length === 1 ? "rule" : "rules"} loaded.`;
-    yield* Console.log([loaded, formatContradictions(contradictions)].join("\n"));
+    const tallied = yield* tallyProjectRules(entries, config.model);
+    // The wording and the tallies need no request, so they show before Jev is asked anything.
+    yield* Console.log(
+      [
+        loaded,
+        ...formatStrays(straysAmong(entries), root),
+        ...formatLinterCheck(tallied, root),
+      ].join("\n"),
+    );
+    const contradictions = yield* findContradictions(entries, config.threshold);
+    yield* Console.log(formatContradictions(contradictions));
     if (contradictions.length > 0) {
       return yield* ContradictionsReported.make({ count: contradictions.length });
     }
   }),
 ).pipe(
   Command.withDescription(
-    "Load the config, rule files, and presets the way lint does, then ask Jev whether any two rules that apply to the same files contradict each other.",
+    "Load the config, rule files, and presets the way lint does, check each rule's wording against the README's rule writing tips, report the rules lint's linter check flags, then ask Jev whether any two rules that apply to the same files contradict each other.",
   ),
   Command.provide((input) => validateLayer({ presets: chosenPresets(input.preset) })),
 );
@@ -312,7 +333,10 @@ export const cli = Command.make("adhere").pipe(
   Command.withDescription("Lint a repository against the rules in .adhere/, judged by Jev."),
   Command.withExamples([
     { command: "adhere init", description: "Scaffold .adhere/config.ts and two example rules" },
-    { command: "adhere validate", description: "Check the config and rules without calling Jev" },
+    {
+      command: "adhere validate",
+      description: "Check the rules' wording, and ask Jev whether any contradict",
+    },
     { command: "adhere login", description: "Save your TypeSafe AI API key for later runs" },
     { command: "adhere lint", description: "Audit the working directory" },
     {