@drkmttr/adhere 0.13.0 → 0.14.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -3,61 +3,370 @@
3
3
  <sup>A linter for non-deterministic rules. Powered by Typesafe</sup>
4
4
  </div>
5
5
 
6
- A linter for rules a normal linter cannot check. Describe rules using RFC2119
7
- language, like "business logic MUST live in services", or "A 3rd-party API going down
8
- SHOULD NOT also take us down". A rule also includes a good example, a bad example, or
9
- both (which is highly recommended according to [our studies](eval/studies/example-count.md).)
6
+ A linter for rules a normal linter cannot check, such as "Business logic should live in services", or "modules should expose lots of functionality behind a small API". Define rules as formatted markdown:
10
7
 
11
- For each source file, adhere asks [Jev](https://typesafe.ai) (TypeSafe AI's System One
12
- model) whether the file breaks each rule, gets a calibrated probability per
13
- rule, and reports the ones above a threshold with the section of the file Jev
14
- points at. The report uses the same frame as `vp lint`.
8
+ ````md
9
+ ---
10
+ description: A third party API going offline should not also take our app down.
11
+ ---
15
12
 
16
- Here's a demo of what the output looks like:
13
+ Our APIs should never be coupled to some third party API on a critical path. The user facing
14
+ side should be unaffected if it goes offline, and ingestion should happen in a separate asynchronous process.
17
15
 
18
- <a href="https://asciinema.org/a/1266511" target="_blank"><img width="400" src="https://asciinema.org/a/1266511.svg" /></a>
16
+ ## Should
17
+
18
+ ```ts
19
+ import googleMaps from "./lib/maps"
20
+ import db from "./lib/db"
21
+ import redis from "redis"
22
+ import cron from "node-cron"
23
+
24
+ // asume this is called by /api/restaurants/:zipcode
25
+ export async function findRestaurants(zipCode: number) {
26
+ const results = db.restaurants.find({ where: { zipCode }})
27
+ return results
28
+ }
19
29
 
30
+ export async startWorker() {
31
+ cron("0 0 * * *", async () => {
32
+ const since = Number(await redis.get("maps:sync:timestamp"))
33
+ const items = await googlemaps.findAll({ since, limit: 100 })
34
+ const syncItem = item => db.restaurants.upsert(item)
35
+ const next = items.sort((a, b) => b.id < a.id).at(-1)?.timestamp
36
+ const touch = () => redis.set("maps:sync:timestamp", next ?? Date.now())
20
37
 
21
- Rules a normal linter can check exactly, such as a banned import or a type
22
- error, belong in that linter. While it lints, adhere asks Jev whether a normal
23
- linter could check each of your rules, and [`adhere validate`](#validate) lists
24
- rules that it thinks belong in a regular linter, and also detects any
25
- contradictions in your rules.
38
+ return Promise.all(items.map(item => syncItem(item))).then(touch)
39
+ })
40
+ }
41
+ ```
42
+
43
+ ## Never
44
+
45
+ ```ts
46
+ import googleMaps from "./lib/maps";
47
+ import db from "./lib/db";
48
+
49
+ // asume this is called by /api/restaurants/:zipcode
50
+ export async function findRestaurants(zipCode: number) {
51
+ // what happens when google maps goes down? how about latency? how about costs?
52
+ const results = await googleMaps.findAll({ where: { zipCode }, limit: 100 });
53
+ return results;
54
+ }
55
+ ```
56
+ ````
57
+
58
+ Your rules are parsed, then adhere asks [Jev](https://typesafe.ai) (TypeSafe AI's
59
+ System One model) whether the file breaks each rule, gets a calibrated
60
+ probability per rule, and reports the ones above a threshold with the section of
61
+ the file Jev points at. The report uses the same frame as `vp lint`.
62
+
63
+ Here's a demo of what the output looks like.
26
64
 
27
- Presets for TypeScript, React, security, Effect, and alchemy are included, and run without setup:
65
+ <details>
66
+ <summary>
67
+ <h3>Demo 🎬</h3>
68
+ </summary>
69
+ <img alt="demo" align="center" src="https://github.com/darkmatter/adhere/raw/main/eval/demo.gif" />
70
+ </details>
71
+
72
+ Presets for TypeScript, React, security, Effect, and alchemy are included, and
73
+ run without setup:
28
74
 
29
75
  ```sh
30
76
  TYPESAFE_API_KEY=xxx npx @drkmttr/adhere lint --preset typescript
31
77
  ```
32
78
 
79
+ Rules a normal linter can check exactly, such as a banned import or a type
80
+ error, belong in that linter. While it lints, adhere asks Jev whether a normal
81
+ linter could check each of your rules. [`adhere validate`](#check-for-contradictions) lists the
82
+ ones it thinks belong in a regular linter, and detects contradictions between
83
+ your rules.
84
+
85
+ ## Contents
86
+
87
+ - [Install](#install)
88
+ - [Quick start](#quick-start)
89
+ - [Generate rules from your repo](#generate-rules-from-your-repo)
90
+ - [Install rules from other repos](#install-rules-from-other-repos)
91
+ - [Check for contradictions](#check-for-contradictions)
92
+ - [Live evals](#live-evals)
93
+ - [Writing good rules](#writing-good-rules)
94
+ - [Commands](#commands)
95
+ - [Reading the report](#reading-the-report)
96
+ - [Rule format](#rule-format)
97
+ - [Config](#config)
98
+ - [Running lint](#running-lint)
99
+ - [Comments and suppressions](#comments-and-suppressions)
100
+ - [Cache](#cache)
101
+ - [Agent skills](#agent-skills)
102
+ - [How a file is judged](#how-a-file-is-judged)
103
+ - [Upgrading](#upgrading)
104
+ - [Development](#development)
105
+ - [License](#license)
106
+
33
107
  ## Install
34
108
 
35
109
  ```sh
36
- bun add --global @drkmttr/adhere # or: npm install --global @drkmttr/adhere
110
+ curl -fsSL --create-dirs -o ~/.local/bin/adhere \
111
+ https://github.com/darkmatter/adhere/releases/latest/download/adhere-$(uname -s)-$(uname -m)
112
+ chmod +x ~/.local/bin/adhere
37
113
  ```
38
114
 
39
- To pin the version in a repo, for CI or scripts, add it as a dev dependency
40
- and run `npx adhere`.
115
+ - `adhere` is a single executable for macOS and Linux (arm64 and x64) and for
116
+ Windows (x64), and needs nothing else installed, Node included. Any directory
117
+ on your `PATH` works in place of `~/.local/bin`. On Windows, download
118
+ `adhere-Windows-x86_64.exe` from the
119
+ [latest release](https://github.com/darkmatter/adhere/releases/latest).
120
+ - Run the command again to update. `download/v0.14.0` in place of
121
+ `latest/download` fetches that version.
122
+ - To pin the version in a JavaScript repo, for CI or scripts, add
123
+ `@drkmttr/adhere` as a dev dependency and run `npx adhere`. That `adhere`
124
+ starts through Node, or through Bun under `bunx`.
125
+ - adhere sends each file it judges to Jev at `api.typesafe.ai`, authenticated
126
+ with a TypeSafe AI API key.
41
127
 
42
- `adhere` is a prebuilt executable for macOS and Linux on arm64 and x64 and for
43
- Windows on x64.
128
+ ### Login
44
129
 
45
- adhere sends each file it judges to Jev at `api.typesafe.ai`, authenticated
46
- with a TypeSafe AI API key: the one `adhere login` saved, or
47
- `TYPESAFE_API_KEY` when it is set.
130
+ ```sh
131
+ adhere login # prompts for the key, masking what you type
132
+ adhere login < key.txt # reads it from stdin instead
133
+ adhere logout # deletes the saved key
134
+ ```
135
+
136
+ - The key is saved to `~/.config/adhere/credentials.json`, or under
137
+ `$XDG_CONFIG_HOME` when that is set, readable only by you.
138
+ - `TYPESAFE_API_KEY`, when set, takes precedence over the saved key, so CI can
139
+ pass a key without a login.
140
+ - The key is read only when a request is about to be sent. A run where every
141
+ file is cached needs no key and no network.
48
142
 
49
143
  ## Quick start
50
144
 
51
145
  ```sh
52
- adhere init # .adhere/config.ts, two example rules, and the dependency
53
- adhere login # save your TypeSafe AI API key, once
54
- adhere validate # Checks your rules' wording, and contradictions using Jev
55
- adhere lint # audit the working directory
56
- git add .adhere # commit the rules, and the judgments they cost
146
+ adhere init # .adhere/config.ts, two example rules, and the dependency
147
+ adhere login # save your TypeSafe AI API key, once
148
+ adhere validate # check your rules' wording, and ask Jev whether any contradict
149
+ adhere lint # audit the working directory
150
+ git add .adhere # commit the rules, and the judgments they cost
151
+ ```
152
+
153
+ ### Init
154
+
155
+ `adhere init` creates a config file, some example rules, and adds adhere as a
156
+ dev dependency.
157
+
158
+ It is safe to rerun: existing files are reported as skipped, and only `--force`
159
+ overwrites them.
160
+
161
+ ## Generate rules from your repo
162
+
163
+ Let an agent write your first rules with you. The
164
+ [`adhere-setup`](./skills/adhere-setup/SKILL.md) skill walks it through setting
165
+ adhere up in your repo:
166
+
167
+ 1. It reads where your conventions live: `AGENTS.md`, `CONTRIBUTING.md`, docs
168
+ and ADRs, your lint config, recent review comments, and the code others copy
169
+ from.
170
+ 2. It drafts the conventions a normal linter cannot check as rules, each with
171
+ real code from your repo, and sets aside the ones a linter could check.
172
+ 3. It adds the presets that fit your stack, and rules from your organization's
173
+ [shared repo](#install-rules-from-other-repos), if you have one.
174
+ 4. It shows every candidate in one list, with its evidence, and imports the ones
175
+ you choose.
176
+ 5. It validates them, walks you through the first lint and what it costs, and
177
+ adds adhere to CI.
178
+
179
+ It stops to ask you at each of those decisions, and never asks for your API key.
180
+
181
+ ```sh
182
+ skills add darkmatter/adhere # install adhere's skills for your agent
183
+ ```
184
+
185
+ Then ask your agent to set adhere up.
186
+
187
+ ## Install rules from other repos
188
+
189
+ An organization's rules can live in one repo's `.adhere/rules/`, and other repos
190
+ copy them in with `adhere install`:
191
+
192
+ ```sh
193
+ adhere list darkmatter/standards # each rule there, with its description
194
+ adhere install darkmatter/standards # every rule
195
+ adhere install darkmatter/standards/data # the data topic
196
+ adhere install darkmatter/standards/data/brand-ports # one rule
197
+ adhere install darkmatter/standards#v3 # every rule, as tagged v3
57
198
  ```
58
199
 
59
- Without a config or rules, `adhere lint --preset effect` audits against the
60
- built-in Effect rules. A finding looks like this:
200
+ - Copies land in this repo's `.adhere/rules/org/repo/`. So `data/brand-ports`
201
+ from darkmatter/standards is
202
+ `.adhere/rules/darkmatter/standards/data/brand-ports/`, the rule
203
+ `darkmatter/standards/data/brand-ports`, and two repos' rules never collide.
204
+ - The copies are the repo's own rules from then on: commit them, edit them, or
205
+ start new rules from them. Nothing tracks where they came from.
206
+ - A later install skips a rule already there, as init does. `--force` replaces
207
+ its directory, edits and all.
208
+ - A rule's whole directory is copied. The source's config, its inline rules, and
209
+ its cache are not.
210
+
211
+ > **Read what you install.** A `RULE.ts` is copied with the helpers beside it,
212
+ > and runs whenever lint does, in CI too. `adhere list` names a `RULE.ts`
213
+ > without its description, since reading it would run it.
214
+
215
+ The source is cloned with `git` from `https://github.com/org/repo.git`, so a
216
+ private repo needs git's credentials for GitHub, as `gh auth setup-git` sets up.
217
+ To clone over SSH instead, let git rewrite the URL:
218
+ `git config --global url.git@github.com:.insteadOf https://github.com/`.
219
+
220
+ ### Creating a shared repo
221
+
222
+ `adhere init --shared org/repo` scaffolds a repo whose rules other repos copy in
223
+ with `adhere install`, instead of one that lints itself. It writes the two
224
+ example rules and no config, since install copies only rule files, and with
225
+ them:
226
+
227
+ | File | What it is for |
228
+ | ------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
229
+ | `README.md` | Says what the repo is for, how to install its rules, and how to add one. |
230
+ | `.github/workflows/adhere.yaml` | Runs `adhere validate` on every push to main and every pull request, with the `TYPESAFE_API_KEY` secret. A pull request from a fork gets no secrets, so validate refuses there. |
231
+ | `alchemy.run.ts` | An [alchemy](https://alchemy.run) stack that creates the GitHub repo, or adopts it, and sets that secret from `TYPESAFE_API_KEY` when deployed with `npx alchemy deploy`. |
232
+ | `package.json`, `.gitignore` | alchemy and effect, pinned to versions that work together, which init installs, and a `.gitignore` for them and alchemy's state. |
233
+
234
+ The repo is public unless you answer yes when init asks, at a terminal, whether
235
+ to make it private. Other repos then need git's credentials for it to list and
236
+ install its rules. Without a terminal it is public: change `visibility` in the
237
+ stack to make it private.
238
+
239
+ ## Check for contradictions
240
+
241
+ Two rules contradict when no code can follow both, such as one that says errors
242
+ must be thrown and one that says they must be returned. Each rule reads fine on
243
+ its own, so a contradiction shows up only as code that breaks one rule or the
244
+ other whatever you do. `adhere validate` finds them before lint does:
245
+
246
+ ```sh
247
+ adhere validate # your rules
248
+ adhere validate --preset effect # your rules beside a preset's
249
+ ```
250
+
251
+ ```text
252
+ Found 1 contradiction among configured rules:
253
+
254
+ 1. No code can follow both (0.91):
255
+ - errors/throw-tagged-errors (/repo/.adhere/rules/errors/throw-tagged-errors/RULE.md)
256
+ A function that fails must throw a tagged error, never return an error value.
257
+ - errors/return-results (/repo/packages/api/.adhere/rules/errors/return-results/RULE.md)
258
+ A function that fails must return a Result, never throw.
259
+
260
+ Resolve by editing one rule, narrowing a nested rule's scope, or using the same rule id when the nested rule is meant to shadow the root rule.
261
+ ```
262
+
263
+ It asks Jev about each pair of rules that apply to the same files, so it needs
264
+ the API key, as `lint` does. A contradiction fails it, with exit code 1, so it
265
+ can gate a pull request that adds a rule, as the workflow
266
+ [`adhere init --shared`](#creating-a-shared-repo) writes does.
267
+
268
+ It also runs two checks that are advice and never fail it:
269
+
270
+ - **Wording**: each rule worded otherwise than the
271
+ [rule writing tips](#rule-writing-tips) recommend, with its file and how.
272
+ - **The linter check**: while `lint` judges a file against one of your rules, it
273
+ also asks Jev whether a linter or type checker could decide the rule exactly,
274
+ on 10 files per rule. A rule flagged on 7 or more of them probably belongs in
275
+ a regular linter. One flagged on 3 to 6 reads differently from file to file,
276
+ so its description should say more precisely what it applies to. Preset rules
277
+ are left out.
278
+
279
+ Two rules with the same id are never compared: a nested rule that shares an id
280
+ shadows the other on purpose.
281
+
282
+ ## Live evals
283
+
284
+ Try a change to a rule on your own code before you rely on it: run the current
285
+ rule and the changed one side by side, and compare what each flags.
286
+
287
+ 1. Copy the rule's directory under a new id:
288
+
289
+ ```sh
290
+ cp -r .adhere/rules/data/brand-ports .adhere/rules/data/brand-ports-next
291
+ ```
292
+
293
+ 2. Give the copy `level: warning` in its front matter. Its findings show in
294
+ amber and never fail the run, so the experiment can sit in the repo, and in
295
+ CI, while you watch it.
296
+ 3. Change the copy: its description, its examples, its `appliesTo` or
297
+ `excludeIf`, or what it [`reads`](#what-a-rule-reads). To try giving Jev more
298
+ to read, make the copy a `RULE.ts` with an
299
+ [`appendState`](#rules-as-typescript-files) hook.
300
+ 4. Run `adhere lint`, or `adhere lint --filter '<glob>'` to try it on part of
301
+ the repo first, and compare the two rules' findings on the same files.
302
+
303
+ Judge the two by recall, the share of real violations each catches, and
304
+ precision, the share of its flags that are real. Check the findings by hand, or
305
+ with the [`adhere-fix`](./skills/adhere-fix/SKILL.md) skill, which verifies each
306
+ one against its rule and the code. Then keep the better version under the
307
+ original id, and delete the other.
308
+
309
+ - **The copy costs nothing until you change it.** Judgments are cached by the
310
+ rule's text, not by its id, its level, or its threshold, so a copy that reads
311
+ the same as the original answers from the original's cache. Each change after
312
+ that judges the copy, and only the copy, once per file.
313
+ - **Keeping the winner is free.** Moving the winning text to the original id
314
+ reuses its cached judgments.
315
+ - **Counting each rule's findings** takes a pipe:
316
+ `adhere lint --yes | grep -c 'data/brand-ports-next:'`.
317
+
318
+ ## Writing good rules
319
+
320
+ We've evaluated how Jev reads a rule, on the presets and on other repos, to
321
+ catch the most violations with the fewest false positives. The
322
+ [eval](eval/README.md#studies) has every study; these are the lessons for
323
+ writing a rule.
324
+
325
+ ### Rule writing tips
326
+
327
+ - **Say "must" and "never"**, in the description and in the headings over the
328
+ code, or "should" and "should not" for a guideline. Rewording the presets'
329
+ descriptions this way, with a bad example per rule, was the largest gain of
330
+ any study: at 0.8, as many caught, and 1 wrong where there had been 5 or 6
331
+ ([study](eval/studies/must-never.md)).
332
+ - **Give one example of each kind**: one `must` block and one `never` block. A
333
+ bad example helped every wording, and at 0.9 caught 24 violations with none
334
+ wrong, where there had been 17
335
+ ([study](eval/studies/rule-vocabulary.md)). Either kind alone did worse, and
336
+ three of each did no better, for 1.58 times the tokens
337
+ ([study](eval/studies/example-count.md)).
338
+
339
+ `adhere validate` lists each rule worded otherwise.
340
+
341
+ ### What else the studies found
342
+
343
+ | Lesson | What the study found |
344
+ | ------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
345
+ | "Should" is for people, not a weaker check. | Rules written with "should" and with "must" scored nearly the same ([study](eval/studies/rule-vocabulary.md)). |
346
+ | A wrong finding is usually the rule's fault. | Above 0.8, 31 of 35 wrong findings repeated within a rule: a scope its words reach past, or the same misreading again. Fixing the wording fixes them together. Below 0.8, one-off misreads grow, to about one finding in six at 0.6 to 0.7 ([study](eval/studies/presets.md)). |
347
+ | Narrow a rule in its description. | Jev scores an `excludeIf` near 0.5 for real and false findings alike, so it thins a rule's findings about as much as a higher threshold ([study](eval/studies/presets.md)). Narrowing the description moves the judgment itself: scoping alchemy's idempotent-delete rule to providers' delete handlers dropped its findings elsewhere from 0.67–0.88 to 0.08–0.21, with nothing real lost ([study](eval/studies/idempotent-delete.md)). |
348
+ | Beware rules that turn on what the file does not show. | The weakest preset rules hinged on facts outside the file: whether a client has a default timeout, whether a script is an entry point, whether a key is public ([study](eval/studies/presets.md)). Warning below a context of 0.7 marked one finding in five, and those were real about half the time, where the rest were real three times in four ([study](eval/studies/sufficiency.md)). Make such a rule a warning, or give Jev the fact with [`reads`](#what-a-rule-reads), an [`appendState`](#rules-as-typescript-files) hook, or an `@adhere` note. |
349
+ | Jev believes comments. | Eight of nine real violations of the idempotent-delete rule carried a comment wrongly saying they were fine, and scored 0.31 to 0.62; without comments, all nine scored 0.81 or more ([study](eval/studies/idempotent-delete.md)). That is why adhere takes comments out. A fact Jev needs, such as an exemption, goes in an `@adhere` note, which stays ([study](eval/studies/comments.md)). |
350
+
351
+ ## Commands
352
+
353
+ | Command | What it does |
354
+ | --------------------------------- | ------------------------------------------------------------------------------------------------------------------------- |
355
+ | `adhere lint` | Audit the working directory. See [Running lint](#running-lint). |
356
+ | `adhere validate` | Check the rules' wording, and ask Jev whether any contradict. See [Check for contradictions](#check-for-contradictions). |
357
+ | `adhere init [--force]` | Scaffold `.adhere/config.ts` and two example rules. See [Init](#init). |
358
+ | `adhere init --shared org/repo` | Scaffold a repo of rules other repos install. See [Creating a shared repo](#creating-a-shared-repo). |
359
+ | `adhere list org/repo` | List the rules in another repo's `.adhere/rules/`. See [Install rules from other repos](#install-rules-from-other-repos). |
360
+ | `adhere install org/repo` | Copy them into this repo's `.adhere/rules/org/repo/`. |
361
+ | `adhere login`, `adhere logout` | Save or delete a TypeSafe AI API key. See [Login](#login). |
362
+ | `adhere skill [docs\|setup\|fix]` | Print an agent skill. See [Agent skills](#agent-skills). |
363
+
364
+ Bare `adhere` prints the help. `adhere <command> --help` lists a command's
365
+ flags, and `adhere --completions <shell>` prints a completion script.
366
+
367
+ ## Reading the report
368
+
369
+ A finding looks like this:
61
370
 
62
371
  ```text
63
372
  × data/brand-ports: A port must be a branded, range-checked integer, never a bare number.
@@ -86,39 +395,27 @@ Found 1 error.
86
395
  42 files, 3 judged, 39 cached.
87
396
  ```
88
397
 
89
- The header is the rule id and the rule's description, and two scores have
90
- the next line. `confidence` is Jev's probability that the file breaks the
91
- rule. `context` is its probability that the file shows enough to decide that
92
- (see below). On a terminal a confidence is amber near its rule's threshold,
93
- and green above that. Near is within a quarter of the room above the
94
- threshold, on either side: 0.05 at a threshold of 0.80, so 0.75 to 0.84. A
95
- rule whose findings are mostly amber turns on where its threshold sits, which
96
- may be too low. Below the amber a score is red: lint reports no finding with
97
- a confidence there. A context is colored the same way around the sufficiency
98
- threshold, so at 0.60 it is amber from 0.50 to 0.69 and red below 0.50, the
99
- finding to doubt first. A legend under the last finding gives each score's
100
- threshold between the range in red and the range in green. The threshold and
101
- each `<` are amber, for the scores between them. Confidence's threshold there is the run's;
102
- a rule with a threshold of its own has its ranges around that.
103
- A preset's rule has the preset's name first, as in
104
- `effect/basics/gen-for-sequencing`, so a report that mixes presets
105
- with a repo's own rules says where each came from.
106
- Under it is the section of the file Jev points at, with the line it names
107
- underlined, or the lines, when it names several, marked with a bar beside the
108
- code. adhere splits each file into sections of whole statements, such as
109
- its imports, constants, and functions, packed in order into runs of at most 30
110
- lines; a class or function longer than that splits into its methods or the
111
- statements of its body. Jev reads the file as these sections and chooses one,
112
- and in the same request the line in it where the violation starts and the one
113
- where it ends: on the eval it judged as well that way, and the lines it named
114
- held a violation for 94% of real findings, most often as one line. When it
115
- names no lines, the section is shown alone, at its first line.
116
-
117
- Jev is also asked whether the file shows enough to decide the rule at all:
118
- whether it turns on something the file does not show, such as what another
119
- file, a library, or a service does. Its probability that the file does show
120
- enough is the finding's `context`. When that is below 0.6, the finding gets a
121
- warning under its code, and help on adding what is missing:
398
+ | Part | What it tells you |
399
+ | ------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------- |
400
+ | `×` or `⚠` | An error, or a warning, in amber, from a rule whose `level` is `warning`. Only errors fail the run. |
401
+ | `data/brand-ports` | The rule's id, then its description. A preset's rule has the preset's name first, as in `effect/basics/gen-for-sequencing`. |
402
+ | `confidence` | Jev's probability that the file breaks the rule. |
403
+ | `context` | Jev's probability that the file shows enough to decide that. |
404
+ | The excerpt | The section of the file Jev points at. The line it names is underlined, or the lines, when it names several, are marked with a bar beside the code. |
405
+ | `hint:` | The rule's code that must be written. A rule with only code that must never be written shows that code instead, labeled `never:`. |
406
+ | The legend | Under the last finding: each score's threshold, between its range in red and its range in green. |
407
+
408
+ **Score colors.** On a terminal, each score is colored by where it sits around
409
+ its threshold:
410
+
411
+ - If you're near the threshold (orange), consider adjustments.
412
+ - Red is below the threshold, so lint never reports it: only the legend shows
413
+ that range.
414
+
415
+ **Low context.** Jev sees one file at a time. So we also ask whether the file
416
+ shows enough to decide the rule at all. When `context` is below 0.6, it means
417
+ Jev thinks it needs more info, which you can add with an `@adhere` comment.
418
+ You may also see a hint:
122
419
 
123
420
  ```text
124
421
  warning: this file may not show enough to check this rule
@@ -126,317 +423,277 @@ warning under its code, and help on adding what is missing:
126
423
  // @adhere <the fact>, and how you know it
127
424
  ```
128
425
 
129
- On the eval, most findings with this warning were false, where about one in
130
- five of all findings was: check what the code relies on outside the file
131
- before acting on it. What you find can go in an `@adhere` note, which Jev
132
- reads (see [Comments](#comments)). `sufficiencyThreshold` in the config, or `--sufficiency-threshold`, sets
133
- the cutoff. The warning changes neither the finding's level nor the exit code.
426
+ - On the eval, most findings with this warning were false, where about one in
427
+ five of all findings was ([study](eval/studies/sufficiency.md)). Check what
428
+ the code relies on outside the file before acting on it.
429
+ - If you feel the threshold is too low/high, override it using `sufficiencyThreshold`
430
+ in the config.
134
431
 
135
- The hint is the rule's code that must be written; a rule with only code that must
136
- never be written shows that code, labeled `never:`, instead. A rule whose
137
- level is `warning` reports its findings with `⚠` in place of `×`, in amber. On
138
- a terminal, the report is in color and the code in it is highlighted. The exit
139
- code is 1 when an error is reported, and 0 when nothing is, or only warnings;
140
- `--deny-warnings` fails the run on warnings too. It is also 1 when the run
141
- refuses, for example on an invalid config or rule file, a missing API key, or
142
- an unknown command or flag; a refusal prints its reason.
432
+ ## Rule format
143
433
 
144
- ## Usage
434
+ A repo's own rules live in `.adhere/rules/`, organized just like skills:
145
435
 
146
- ```sh
147
- adhere lint # audit the working directory
148
- adhere lint --preset effect # add a built-in rule set; the config becomes optional
149
- adhere lint --threshold 0.9 # replace the config's threshold; per-rule thresholds still apply
150
- adhere lint --sufficiency-threshold 0.5 # warn on fewer findings that may turn on other files
151
- adhere lint --yes # send the requests without asking first
152
- adhere lint --limit 500 # judge at most 500 checks; the rest wait for the next run
153
- adhere lint --rpm 30 # send at most 30 requests a minute
154
- adhere lint --filter 'src/**' # read only the files a glob matches
155
- adhere lint --deny-warnings # fail on warnings as well as errors
156
- adhere lint --log-level debug # log each request to Jev on stderr
157
- adhere validate # check the rules' wording; ask Jev whether any contradict
158
- adhere init [--force] # scaffold .adhere/config.ts and two example rules
159
- adhere init --shared org/repo # scaffold a repo of rules other repos install
160
- adhere list org/repo # list the rules in another repo's .adhere/rules/
161
- adhere install org/repo # copy them into this repo's .adhere/rules/org/repo/
162
- adhere login # save a TypeSafe AI API key for later runs
163
- adhere logout # delete the saved key
164
- adhere skill [setup|fix] # print an agent skill (below)
436
+ ```text
437
+ .adhere/
438
+ config.ts optional
439
+ cache/ commit it
440
+ rules/
441
+ data/ a topic: a directory that holds no rule
442
+ brand-ports/
443
+ RULE.md the rule data/brand-ports
444
+ columns/
445
+ RULE.ts a rule written in TypeScript
446
+ schema.ts a helper it imports, not read as a rule
165
447
  ```
166
448
 
167
- Bare `adhere` prints the help, which lists the commands, and
168
- `adhere <command> --help` lists a command's flags. `adhere --completions <shell>`
169
- prints a completion script.
449
+ - The directory's path is the rule id: `.adhere/rules/data/brand-ports/RULE.md`
450
+ is `data/brand-ports`.
451
+ - A rule can be markdown `RULE.md`, or typescript [`RULE.ts`](#rules-as-typescript-files). A
452
+ directory with both refuses the run.
453
+ - Files not named `RULE` are ignored.
170
454
 
171
- ### Before and during a run
455
+ ### Rules as Markdown files
172
456
 
173
- Until it knows what judging takes, a status line on stderr says what `lint` is
174
- doing: looking for `.adhere/` rules, listing and reading the files, then
175
- planning, each with a count, such as `Planning: 5,120 of 10,333 files`. The
176
- line is drawn only on a terminal, and not at `--log-level debug` or below,
177
- whose log lines would land in it. Then `lint` says on stderr what it found and
178
- what judging takes:
457
+ A `RULE.md` is front matter, then a body that holds the rule's code:
179
458
 
180
- ```text
181
- 200 files and 14 rules: 2800 checks, 1400 cached.
182
- Judging the other 1400 takes 200 requests to Jev, plus 1 or more for each file with a finding.
183
- Those carry about 1.9 million input tokens: about $0.08 at $0.042 per million, and more for locating findings.
184
- ? Send 200 requests to Jev, about $0.08? › (Y/n)
459
+ ````md
460
+ ---
461
+ description: A port must be a branded, range-checked integer, never a bare number.
462
+ threshold: 0.8
463
+ ---
464
+
465
+ Why: a bare `number` accepts 70000 and -1.
466
+
467
+ ## Must
468
+
469
+ ```ts
470
+ const Port = Schema.Int.pipe(
471
+ Schema.check(Schema.isBetween({ minimum: 1, maximum: 65535 })),
472
+ Schema.brand("Port"),
473
+ );
185
474
  ```
186
475
 
187
- The cost is adhere's estimate of the input tokens times the model's price.
188
- Jev charges only for input tokens: $0.042 a million for `jev-latest`, per
189
- [TypeSafe AI's models page](https://docs.typesafe.ai/models) in September 2026.
190
- For a model adhere has no price for, the plan gives the tokens alone.
476
+ ## Never
191
477
 
192
- It asks only with a terminal on stdin and stdout, and never when the cache
193
- answers every check; `--yes` sends without asking. Files finished before it are cached, so
194
- running again continues from there.
478
+ ```ts
479
+ const port: number = Number(process.env.PORT);
480
+ ```
481
+ ````
195
482
 
196
- The firewall in front of Jev's API can refuse a request whose code reads to
197
- it as an attack, with a 403 and an HTML page rather than Jev's JSON. It refuses
198
- the same request every run, so that file is skipped, the run goes on, and the
199
- report lists each such file with Cloudflare's Ray ID, which TypeSafe AI can
200
- look the block up by. When only the question that finds the line is refused,
201
- the file's judgments are kept, and a rerun sends that question alone.
483
+ | Front matter | Meaning |
484
+ | ------------------------ | ---------------------------------------------------------------------------------------------------------------------------------------------- |
485
+ | `description` | Required. One sentence saying what code must be, and never be. |
486
+ | `threshold` | The rule's own cutoff, in place of the config's. |
487
+ | `level` | `warning` reports the rule's findings as warnings, which do not fail the run. Use it for a nit, or for a rule that tends to flag code wrongly. |
488
+ | `tests` | Rules skip tests by default. `only` is for a rule about tests, which judges nothing else. `include` is for one that holds in tests as well. |
489
+ | `appliesTo`, `excludeIf` | Where the rule applies. See [Scoping a rule](#scoping-a-rule). |
490
+ | `reads` | What Jev reads beside the code. See [What a rule reads](#what-a-rule-reads). |
491
+
492
+ | Heading in the body | The code under it |
493
+ | ---------------------------- | ----------------------------------------------------------- |
494
+ | `## Must` | must be written |
495
+ | `## Never` | must never be written, which is what a violation looks like |
496
+ | `## Should`, `## Should not` | the same, for a guideline |
497
+
498
+ - A rule needs code under at least one heading.
499
+ - A rule is a requirement ("must", "never") or a guideline ("should", "should
500
+ not"), and cannot mix the two.
501
+ - A rule with only a `never` block suits a rule with no single correct form to
502
+ show, such as a hand-rolled retry loop or an error caught and dropped.
503
+ - Neither goes in the other's place: code that must never be written, under
504
+ `must`, reads to Jev as the pattern to follow.
505
+ - Prose around the code is the rule's details: Jev reads it after the
506
+ description, so use it to say why the rule holds or where it does not apply.
507
+ The report shows the description alone.
508
+
509
+ <details>
510
+ <summary>More on headings, fences, and loading</summary>
511
+
512
+ - A heading names the code in its section, which runs to the next heading at its
513
+ level or higher, so a deeper heading such as `### A bare number` stays inside
514
+ it.
515
+ - Only a heading that is the word alone names code, in any case and with or
516
+ without a colon: `## Never:` does, `## Never do this` does not.
517
+ - A fence can instead name its code after its language, as in `ts never`, which
518
+ GitHub does not show. A fence's own word wins over its heading's.
519
+ - A rule inline in a config takes the same keys, with its code under `must`,
520
+ `never`, `should`, and `shouldNot`, and its prose under `details`.
521
+ - `loadRules(directory)` from the package root does the same load for your own
522
+ tooling.
523
+
524
+ </details>
202
525
 
203
- ### Limiting a run
526
+ ### Scoping a rule
204
527
 
205
- Three flags bound what a run does. `--limit <checks>` judges at most that many
206
- checks, taken in path order; the rest wait, and since judgments are cached, the
207
- next run with the same limit picks up where this one stopped. `--limit 0` shows
208
- the plan and judges nothing. `--rpm <requests>` sends at most that many requests
209
- to Jev a minute, evenly spaced, retries included, and the plan says about how
210
- long they take. `--filter <glob>` reads only the files whose path from the
211
- working directory matches, such as `src/**` or `**/*.service.ts` (if using globs,
212
- wrap with single quotes to avoid expansion); repeat it for more, and start a
213
- pattern with `!` to leave out what it matches. Together:
528
+ `appliesTo` and `excludeIf` say where a rule applies, apart from what it asks
529
+ for. Each is a list of descriptions of code: a JSON array on one line in front
530
+ matter, or an array in a config or a TypeScript rule.
214
531
 
215
- ```sh
216
- adhere lint --filter 'packages/api/**' --filter '!**/generated/**' --limit 200 --rpm 30
532
+ <!-- prettier-ignore -->
533
+ ```md
534
+ ---
535
+ description: Structured variants must be Schema.TaggedClass members of a Schema.Union, never hand-written object types joined by a tag field.
536
+ appliesTo: ["a declared union type or schema"]
537
+ excludeIf: ["a type that mirrors a third-party format whose tag key that format fixes, such as a Slack Block Kit block"]
538
+ ---
217
539
  ```
218
540
 
219
- ### Comments
541
+ | Key | Jev is asked, of the code that breaks the rule | A finding stands when |
542
+ | ----------- | ---------------------------------------------- | ------------------------- |
543
+ | `appliesTo` | whether any of it is as described | Jev says yes to every one |
544
+ | `excludeIf` | whether all of it is as described | Jev says yes to none |
220
545
 
221
- Jev reads a file's code, not its comments. A comment changes nothing the code
222
- does, so it should not change the report, yet Jev takes what comments claim at
223
- their word: in a study of one alchemy rule, comments saying a delete succeeds
224
- on a missing resource hid 8 of its 9 real violations, and taking comments out
225
- found all nine. adhere takes every comment out before it hashes or sends a
226
- file, blanking it so every line keeps its number; the report's excerpts still
227
- show them.
546
+ The questions are about the code that breaks the rule, not the whole file, so a
547
+ file with a real violation beside code an `excludeIf` describes keeps its
548
+ finding.
228
549
 
229
- A comment that says `@adhere` anywhere in it is a note for Jev, and stays.
230
- Write one for a fact the code relies on that the file cannot show, once you
231
- have checked it, with how you know:
550
+ A yes is a probability above 0.5. At `--log-level debug`, lint logs each
551
+ finding its matchers dropped, with their scores.
232
552
 
233
- ```ts
234
- /**
235
- * Deletes the activity. @adhere DeleteActivity succeeds on a missing
236
- * activity, so no not-found error needs catching; probed 2026-09-25.
237
- */
238
- ```
553
+ > **Narrowing the description often works better.** In the preset study, Jev
554
+ > scored an `excludeIf` near 0.5 for real and false findings alike. See
555
+ > [Writing good rules](#writing-good-rules).
239
556
 
240
- A note informs Jev; it suppresses nothing, and Jev still judges the code
241
- around it. `includeComments: true` in the config sends every comment, for a
242
- project whose rules are about comments, such as doc comments on exports.
557
+ ### What a rule reads
243
558
 
244
- ### Suppressing a finding
559
+ Jev judges a file by its code alone. A rule that turns on something the file
560
+ does not show can name it in `reads`, and adhere gives it to Jev beside the
561
+ code, for that rule only. It is a JSON array on one line in front matter, or an
562
+ array in a config or a TypeScript rule.
245
563
 
246
- A finding Jev got wrong is suppressed in the code, with a comment saying why:
564
+ ````md
565
+ ---
566
+ description: A function from one of this repository's own packages, which `workspacePackages` names, must be called through the package's namespace, never imported by its own name.
567
+ reads: ["workspacePackages"]
568
+ ---
569
+
570
+ ## Must
247
571
 
248
572
  ```ts
249
- // adhere-ignore alchemy/providers/idempotent-delete -- DeleteActivity succeeds on a missing activity; probed 2026-09-25
250
- delete: Effect.fn(function* ({ output }) {
251
- yield* sfn.deleteActivity({ activityArn: output.activityArn });
252
- }),
573
+ import * as Orders from "orders-core";
253
574
  ```
254
575
 
255
- On a line of its own, `adhere-ignore` covers the statement that starts on the
256
- next line of code: here the whole handler, wherever in it Jev points, since the
257
- lines it points at can move between runs. A finding without lines is covered
258
- when its section overlaps the statement. At the end of a line of code, it
259
- covers the statement that starts on that line. `adhere-ignore-file`, anywhere
260
- in a file, covers the whole file, and the rules it names are not judged there
261
- at all. A comment names rules as a report does, a preset's with its preset
262
- first, separated by commas; what follows `--` is the reason, for whoever
263
- reads the code next. Jev never reads these comments, even with
264
- `includeComments: true`, and the summary counts the findings they suppress.
576
+ ## Never
265
577
 
266
- ### Logging a run
578
+ ```ts
579
+ import { parse } from "orders-core";
580
+ ```
581
+ ````
267
582
 
268
- `--log-level debug` logs a run's work on stderr: the config it loaded, how many
269
- paths it listed and how many of them it reads, and each request to Jev, with
270
- the file it is for, about how many tokens it carries, the HTTP status, how long
271
- it took, and Cloudflare's Ray ID. A failed attempt is logged even when a retry
272
- hides it. `--log-level trace` adds each file as it is read, planned, and
273
- answered from the cache, and the first 4000 characters of any error Jev's API
274
- answers with. The counter stays off at these levels, since its redraws would
275
- garble the lines, and stdout still carries only the report, so the log can go
276
- to a file of its own:
583
+ There is one name so far:
277
584
 
278
- ```sh
279
- adhere lint --log-level debug 2> adhere.log
280
- ```
585
+ | Name | What Jev reads |
586
+ | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
587
+ | `workspacePackages` | The workspace's packages, each one's name with its directory, as in `{ "orders-core": "packages/orders" }`. A package that is not in it is an installed one. It is for a rule about whose package an import is of: the repository's own, or one installed from a registry. |
281
588
 
282
- ### What gets read
589
+ - Each name is the key Jev reads it under, so the description can name it in
590
+ backticks, as the questions name `code`.
591
+ - A name adhere does not have refuses the run, so a misspelled one does not go
592
+ unnoticed.
593
+ - What adhere has no name for, such as a file of the repository's, a rule in
594
+ TypeScript adds with [`appendState`](#rules-as-typescript-files).
283
595
 
284
- `adhere lint` reads the TypeScript files under the working directory, `.ts`,
285
- `.tsx`, `.mts`, and `.cts`, except declaration files such as `.d.ts` and
286
- config files. When the working directory contains `agents/`, `apps/`, or
287
- `packages/`, only those trees are read. Below the working directory, anything
288
- under `node_modules/`, `dist/`, `coverage/`, `vendor/`, `e2e/`, `references/`,
289
- `.adhere/`, `.agents/`, `.claude/`, `.direnv/`, `.alchemy/`, or `.vite/` is
290
- skipped; the directories above it do not count. JavaScript files are not read,
291
- and `.gitignore` is not consulted.
292
-
293
- Tests are judged only by rules about tests. A test is a `.test` or `.spec`
294
- file, such as `a.test.ts` or `Button.spec.tsx`; a fixture or test helper named
295
- as one, such as `fixtures.ts`, `GitHubHttpFixtures.ts`,
296
- `ledger-copy.fixture.ts`, `linear-test-helpers.ts`, or `test-utils.ts`; or any
297
- file under a `test/`, `tests/`, `__tests__/`,
298
- `testing/`, `fixtures/`, `fixture/`, `__fixtures__/`, or `__mocks__/`
299
- directory, and only a rule whose front matter says `tests` judges
300
- one (see [Rules as Markdown files](#rules-as-markdown-files)). On alchemy, 38%
301
- of the findings from rules not about tests were in test helpers and fixtures.
302
-
303
- A config's `exclude` lists globs, relative to the working directory, of files
304
- no rule judges, such as generated code. It leaves them out of every run, as
305
- `--filter '!<glob>'` leaves them out of one:
596
+ <details>
597
+ <summary>More on requests and where the packages come from</summary>
306
598
 
307
- ```ts
308
- export default defineConfig({ exclude: ["**/generated/**"] });
309
- ```
599
+ - The packages are read on every run, when a rule reads them, from the working
600
+ directory's `package.json`, whose `workspaces` npm, Bun, and Yarn read, and
601
+ from `pnpm-workspace.yaml`'s `packages`.
602
+ - In a pattern, `*` is any directory and `**` any depth of them, and one that
603
+ starts with `!` leaves out what it matches. Each matched directory's
604
+ `package.json` gives the name, and its directory is given from the working
605
+ directory.
606
+ - Nothing need be installed, so a run in CI reads the same packages as one on a
607
+ laptop.
608
+ - A working directory that is not a workspace's root has none.
310
609
 
311
- ### Init
610
+ </details>
312
611
 
313
- `adhere init` writes `.adhere/config.ts`, the `.adhere/tsconfig.json` that gives
314
- the editor its types (see [Config](#config)), and two example rules. It is safe
315
- to rerun: by default it reports existing files as skipped and does not clobber
316
- them. `--force` overwrites them. A config already at another accepted path is
317
- kept, and no second one is added, even with `--force`.
318
-
319
- When there is a `package.json`, init also adds `@drkmttr/adhere` to its
320
- `devDependencies`, so the config's `defineConfig` has its types in the editor. It runs
321
- `<manager> add -D @drkmttr/adhere` with the package manager whose lockfile
322
- (`bun.lock`, `bun.lockb`, `pnpm-lock.yaml`, `yarn.lock`, `package-lock.json`)
323
- sits beside the `package.json`, or npm when there is none. With pnpm it adds
324
- `--ignore-workspace-root-check`, so the install also works at a workspace root.
325
- It skips the install when `package.json` already lists adhere.
326
-
327
- `adhere init --shared org/repo` scaffolds a repo whose rules other repos copy
328
- in with `adhere install` (see [Sharing rules](#sharing-rules)) instead of one
329
- that lints itself. It writes the two example rules and no config, since
330
- install copies only rule files, and with them:
331
-
332
- - `README.md`, which says what the repo is for, how to install its rules, and
333
- how to add one.
334
- - `.github/workflows/adhere.yaml`, which runs `adhere validate` on every push
335
- to main and every pull request, with the `TYPESAFE_API_KEY` secret. A pull
336
- request from a fork gets no secrets, so validate refuses there.
337
- - `alchemy.run.ts`, an [alchemy](https://alchemy.run) stack that creates the
338
- GitHub repo, or adopts it, and sets that secret from `TYPESAFE_API_KEY`
339
- when deployed with `npx alchemy deploy`. The repo is public unless you
340
- answer yes when init asks, at a terminal, whether to make it private. Other
341
- repos then need git's credentials for it to list and install its rules.
342
- Without a terminal it is public; change `visibility` in the stack to make it
343
- private.
344
- - `package.json`, with alchemy and effect pinned to versions that work
345
- together, which init installs, and a `.gitignore` for them and alchemy's
346
- state.
347
-
348
- ### Sharing rules
349
-
350
- An organization's rules can live in one repo's `.adhere/rules/`, and other
351
- repos copy them in. `adhere list org/repo` prints each rule there with its
352
- description. `adhere install org/repo` copies every rule's directory into this
353
- repo's `.adhere/rules/org/repo/`, so `data/brand-ports` from
354
- darkmatter/standards is `.adhere/rules/darkmatter/standards/data/brand-ports/`,
355
- the rule `darkmatter/standards/data/brand-ports`, and two repos' rules never
356
- collide.
357
- A topic or a rule after the repo copies only that part, and `#` picks a branch
358
- or tag:
612
+ ### Rules as TypeScript files
359
613
 
360
- ```sh
361
- adhere install darkmatter/standards/data # the data topic
362
- adhere install darkmatter/standards/data/brand-ports # one rule
363
- adhere install darkmatter/standards#v3 # every rule, as tagged v3
364
- ```
614
+ A rule's directory can hold it as a `RULE.ts` instead, which default-exports
615
+ `defineRule({...})`, taking the fields a config's inline rule does. What
616
+ TypeScript adds is `appendState`, a hook on what Jev reads for the rule:
365
617
 
366
- The copies are the repo's own rules from then on: commit them, edit them, or
367
- start new rules from them. Nothing tracks where they came from, so a later
368
- install skips a rule already there, as init does, and `--force` replaces its
369
- directory, edits and all. A rule's whole directory is copied, so a `RULE.ts`
370
- comes with the helpers beside it, and runs whenever lint does, in CI too: read
371
- what you install. `adhere list` names a `RULE.ts` without its description,
372
- since reading it would run it. The source's config, its inline rules, and its
373
- cache are not copied. A config with inline `rules` replaces `.adhere/rules/`,
374
- so it replaces copied rules too.
618
+ ```ts
619
+ // .adhere/rules/data/columns/RULE.ts
620
+ import { defineRule } from "@drkmttr/adhere";
375
621
 
376
- The source is cloned with `git` from `https://github.com/org/repo.git`, so a
377
- private repo needs git's credentials for GitHub, as `gh auth setup-git` sets
378
- up. To clone over SSH instead, let git rewrite the URL:
379
- `git config --global url.git@github.com:.insteadOf https://github.com/`.
622
+ export default defineRule({
623
+ description:
624
+ "A query must name only columns its table has in `schema`, and never a column it lacks.",
625
+ must: 'db.select("id", "email").from("users")',
626
+ never: 'db.select("mail").from("users")',
627
+ appendState: async (state, file, Bun) => ({
628
+ schema: await Bun.file("db/schema.sql").text(),
629
+ }),
630
+ });
631
+ ```
380
632
 
381
- ### Validate
382
-
383
- `adhere validate` loads the config, rule files, and presets the way `lint`
384
- does, so a file that fails to decode refuses here too, and it prints how many
385
- rules it loaded. `--preset` adds a built-in rule set, as for `lint`.
386
-
387
- Next it checks each rule's wording against the
388
- [rule writing tips](#rule-writing-tips), which sends nothing, and prints each
389
- rule worded otherwise with its file and how:
390
-
391
- - its description does not say a word its code is under, such as "never"
392
- beside a `never` block;
393
- - it has code to write but no example of code that breaks it;
394
- - its description says the other kind of rule's words: "should" in a
395
- requirement, or "must" or "never" in a guideline.
396
-
397
- A rule with only code that must never be written needs no `must` example, as
398
- [Rules as Markdown files](#rules-as-markdown-files) explains. The tips are
399
- advice, so the wording does not change the exit code.
400
-
401
- Then it reports the linter check, which also sends nothing. While `lint`
402
- judges a file against one of the project's own rules, it asks Jev a second
403
- question beside it: should the rule have been checked by a regular linter, that
404
- is, could a linter or type checker have decided exactly whether this file
405
- follows it? It asks on 10 files per rule, spread over the files it judges, and
406
- keeps the answers in the cache until the rule's text changes. `validate` prints
407
- each rule flagged on 7 or more of its 10 files as one a regular linter should
408
- probably check, and each flagged on 3 to 6 as a rule that reads differently
409
- from file to file, whose description should say more precisely what it
410
- applies to. A rule with fewer than 10 answers is counted as waiting; `lint`
411
- collects them as it judges files, so a repo whose files are all cached collects
412
- them as its files change. Preset rules are left out: they are not the
413
- project's to change. The linter check is advice too.
414
-
415
- It then asks Jev whether any two rules that apply to the same files
416
- contradict, so that no code can follow both: one request names, per rule, the
417
- rule it conflicts with, if any, and each named pair then gets a probability
418
- from a request of its own, holding only those two rules, since other rules
419
- beside them dilute the judgment. A pair above the threshold is printed with
420
- both rule files, and the exit code is 1. Two rules with the same id are not
421
- compared: a nested rule that shares an id shadows the other on purpose. When
422
- no two rules share files nothing is sent; otherwise `validate` needs the API
423
- key, as `lint` does.
633
+ - The hook runs right before each request about the rule goes out. It gets the
634
+ request's state, the file the request is about, and Bun's API, and what it
635
+ returns is spread over the state.
636
+ - The description can name a key the hook adds, in backticks, as the questions
637
+ name `code`.
638
+ - A `RULE.ts` is imported, as a config is, so its code runs on every lint.
639
+ - A rule with a hook goes to Jev in requests of its own, so only that rule reads
640
+ what its hook adds. Each of those requests sends the file again.
641
+ - What the hook reads outside the file is not part of the cache: when
642
+ `db/schema.sql` changes, a file already judged is not judged again until it or
643
+ the rule changes.
644
+
645
+ <details>
646
+ <summary>More on the hook</summary>
647
+
648
+ - The state is what Jev reads beside each question: `code`, the file's sections
649
+ as Jev reads them, under their numbers from 1, as in
650
+ `{ code: { "1": "import …", "2": "export const …" } }`.
651
+ - `file` is the file's absolute `path`, and its `contents` as written, comments
652
+ and all.
653
+ - Nothing the hook returns is checked, so it can change `code` too, or replace
654
+ it: a hook can break its own rule's answers, and adhere does not stop it.
655
+ - It can be async, and one that throws refuses the run, naming the rule. A
656
+ `RULE.ts` that default-exports no rule refuses the run too.
657
+ - The plan counts the rule's requests, but not the tokens a hook adds, which
658
+ are not known until it runs. A request over Jev's context skips the file, as a
659
+ file too long does.
660
+ - A `RULE.ts` can import other files by relative path, such as a helper beside
661
+ it in its directory, but the executable resolves no packages besides
662
+ `@drkmttr/adhere`.
663
+ - `Bun` is typed when `@types/bun` is installed and `.adhere/tsconfig.json`
664
+ lists it, as `"types": ["bun"]`, and is `unknown` otherwise.
665
+ - A config's inline rules can have `appendState` too.
666
+
667
+ </details>
668
+
669
+ ### Where rules live
670
+
671
+ - Rules in the root `.adhere/rules/` apply project-wide.
672
+ - A nested `.adhere/` scopes its rules to the directory that contains it. A rule
673
+ in `packages/api/.adhere/rules/data/brand-ports/RULE.md` has the same id,
674
+ `data/brand-ports`, but applies only to files under `packages/api/`.
675
+ - If a nested rule has the same id as a root rule, the nearest containing
676
+ `.adhere/` shadows the less-specific rule for that subtree. Outside that
677
+ subtree, the root rule still applies.
678
+ - Nested `.adhere/` directories are not discovered under `node_modules/`,
679
+ `dist/`, and the other [skipped directories](#what-gets-read).
680
+
681
+ A repo that would rather keep its rules with the rest of its documentation can
682
+ point `rules` at a directory such as `docs/adhere/`:
424
683
 
425
- ### Login
684
+ ```ts
685
+ export default defineConfig({ presets: ["effect"], rules: "./docs/adhere" });
686
+ ```
426
687
 
427
- `adhere login` prompts for a TypeSafe AI API key, masking what you type, and
428
- saves it to `~/.config/adhere/credentials.json`, or under `$XDG_CONFIG_HOME`
429
- when that is set, readable only by you. Piped input is read instead of a
430
- prompt: `adhere login < key.txt`. `lint` and `validate` use the saved key, but
431
- `TYPESAFE_API_KEY`, when set, takes precedence, so CI can pass a key without a
432
- login. `adhere logout` deletes the saved key.
688
+ That directory holds its rules as `.adhere/rules/` does, a directory per rule,
689
+ and they apply project-wide.
433
690
 
434
691
  ## Config
435
692
 
436
693
  A config file is optional. Without one, adhere reads the rules in
437
694
  `.adhere/rules/` and any `--preset`. A config names presets, sets the model and
438
- thresholds, or gives rules inline. It sits in the working directory of the
439
- repo being audited, at one of these paths (keep one):
695
+ thresholds, or gives rules inline. It sits in the working directory of the repo
696
+ being audited, at one of these paths (keep one):
440
697
 
441
698
  - `.adhere/config.ts`, the default, next to the rules
442
699
  - `adhere.config.ts`
@@ -446,12 +703,12 @@ repo being audited, at one of these paths (keep one):
446
703
  import { defineConfig } from "@drkmttr/adhere";
447
704
 
448
705
  export default defineConfig({
449
- model: "jev-latest", // optional, default "jev-latest"
450
- threshold: 0.8, // optional, default 0.8
451
- sufficiencyThreshold: 0.6, // optional, default 0.6: below it, a finding warns to check outside the file
452
- presets: ["effect"], // optional, built-in rule sets
453
- exclude: ["**/generated/**"], // optional, files no rule judges
454
- overrides: { "effect/basics/instrument-with-pipe": "off" }, // optional, see Presets
706
+ model: "jev-latest",
707
+ threshold: 0.8,
708
+ sufficiencyThreshold: 0.6,
709
+ presets: ["effect"],
710
+ exclude: ["**/generated/**"],
711
+ overrides: { "effect/basics/instrument-with-pipe": "off" },
455
712
  rules: {
456
713
  "data/brand-meaningful-primitives": {
457
714
  description:
@@ -460,464 +717,422 @@ export default defineConfig({
460
717
  const UserId = Schema.String.pipe(Schema.brand("UserId"))
461
718
  type UserId = typeof UserId.Type
462
719
  `,
463
- never: "type UserId = string", // optional: what a violation looks like
464
- threshold: 0.8, // optional per-rule override
720
+ never: "type UserId = string",
721
+ threshold: 0.8,
465
722
  },
466
723
  },
467
724
  });
468
725
  ```
469
726
 
470
- The default export is decoded with Effect `Schema`. An invalid shape refuses
471
- the run. `defineConfig` types the config for the editor and returns it as it
472
- is; `satisfies Config`, with `import type { Config }`, does the same. The
473
- executable supplies `@drkmttr/adhere` to the config whether or not the repo has
474
- the package installed, and the editor's types come from the package, which
475
- `adhere init` adds as a dev dependency. A config can import other files by
476
- relative path, but no packages besides `@drkmttr/adhere`: the executable does
477
- not resolve `node_modules`.
478
-
479
- A tsconfig's globs skip dot directories, so the editor opens
480
- `.adhere/config.ts` outside any project, where it cannot resolve the package's
481
- types: they are reached only by `bundler`, `node16`, or `nodenext` resolution.
482
- `adhere init` writes `.adhere/tsconfig.json`, a project for the config and any
483
- [rules written in TypeScript](#rules-as-typescript-files), with `bundler`
484
- resolution. For a config written by hand, add that file, or include
485
- `.adhere/config.ts` in a tsconfig that resolves the same way. One written
486
- before rules could be TypeScript includes only `config.ts`; make its `include`
487
- `["**/*.ts"]` to give rule files their types.
488
-
489
- `rules` in the config, inline as above or as a directory (below), replaces
490
- `.adhere/rules/`: none of its rules, root or nested, is read then.
727
+ | Key | Default | Meaning |
728
+ | ---------------------- | ---------------- | ---------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
729
+ | `model` | `"jev-latest"` | The model id sent to TypeSafe. |
730
+ | `threshold` | `0.8` | Report a rule when Jev's probability is above this. |
731
+ | `sufficiencyThreshold` | `0.6` | Below it, a finding warns that the file may not show enough. |
732
+ | `presets` | none | Built-in rule sets, or their topics. See [Presets](#presets). |
733
+ | `rules` | `.adhere/rules/` | Rules inline, as above, or a directory (see [Where rules live](#where-rules-live)). Either replaces `.adhere/rules/`: none of its rules, root or nested, is read then. |
734
+ | `exclude` | none | Globs, relative to the working directory, of files no rule judges. |
735
+ | `overrides` | none | Settings for a rule by its id. See [Overrides](#overrides). |
736
+ | `includeComments` | `false` | Send every comment to Jev. See [Comments](#comments). |
737
+
738
+ - An invalid shape refuses the run.
739
+ - A config can import other files by relative path, but no packages besides
740
+ `@drkmttr/adhere`: the executable does not resolve `node_modules`.
741
+
742
+ <details>
743
+ <summary>More on types in the editor</summary>
744
+
745
+ - `defineConfig` types the config for the editor and returns it as it is.
746
+ `satisfies Config`, with `import type { Config }`, does the same.
747
+ - The executable supplies `@drkmttr/adhere` to the config whether or not the
748
+ repo has the package installed. The editor's types come from the package,
749
+ which `adhere init` adds as a dev dependency.
750
+ - A tsconfig's globs skip dot directories, so the editor opens
751
+ `.adhere/config.ts` outside any project, where it cannot resolve the package's
752
+ types: they are reached only by `bundler`, `node16`, or `nodenext` resolution.
753
+ `adhere init` writes `.adhere/tsconfig.json`, a project for the config and any
754
+ [rules written in TypeScript](#rules-as-typescript-files), with `bundler`
755
+ resolution. For a config written by hand, add that file, or include
756
+ `.adhere/config.ts` in a tsconfig that resolves the same way.
757
+
758
+ </details>
491
759
 
492
- ### Rules as Markdown files
493
-
494
- A repo's own rules live in `.adhere/rules/`, a directory per rule, which holds
495
- the rule as a `RULE.md`, or as a `RULE.ts` (see
496
- [Rules as TypeScript files](#rules-as-typescript-files)). The directory's path
497
- is the rule id, so `.adhere/rules/data/brand-ports/RULE.md` is
498
- `data/brand-ports`, and a directory that holds no rule, such as `data/`, is a
499
- topic. Nothing else in a rule's directory, such as notes or a helper its
500
- `RULE.ts` imports, is read as a rule, and a directory with both a `RULE.md` and
501
- a `RULE.ts` refuses the run. When `.adhere/rules/` exists, it is read without
502
- any config.
503
-
504
- Rules used to be any `*.md` file in `.adhere/`, as
505
- `.adhere/data/brand-ports.md`. A `*.md` file outside every rule's directory,
506
- but a `README.md`, now refuses the run, and the refusal says where each such
507
- file goes to keep its id, so `adhere-ignore` comments and `overrides` still
508
- name it.
509
-
510
- A `RULE.md` is front matter, then a body. The body's code goes in fences under
511
- headings of RFC 2119's words: `## Must` for code that must be written, and
512
- `## Never` for code that must never be, which is what a violation looks like.
513
- A heading names the code in its section, which runs to the next heading at its
514
- level or higher, so a deeper heading such as `### A bare number` stays inside
515
- it. Only a heading that is the word alone names code, in any case and with or
516
- without a colon: `## Never:` does, `## Never do this` does not. Prose around
517
- the code renders on GitHub and is ignored.
518
-
519
- A fence can instead name its code after its language, as in `ts never`, which
520
- GitHub does not show; a fence's own word wins over its heading's. An untagged
521
- fence outside those sections is code that must be written, and without a fence
522
- the whole body is. `avoid`, the tag before 0.7, reads as `never`.
523
-
524
- ````md
525
- ---
526
- description: A port must be a branded, range-checked integer, never a bare number.
527
- threshold: 0.8
528
- ---
760
+ ### Presets
529
761
 
530
- Why: a bare `number` accepts 70000 and -1.
762
+ | Preset | Rules | For | What it asks for |
763
+ | ------------------------------------- | ----- | ----------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
764
+ | [`typescript`](./presets/typescript/) | 11 | any TypeScript project | Data from outside checked at runtime, invalid states unrepresentable, errors never swallowed, resources released on every path, arguments not mutated, and tests that assert outcomes and stand alone. |
765
+ | [`react`](./presets/react/) | 10 | components and hooks | Logic for an event in its handler rather than an effect, effects that clean up and ignore stale fetches, `useSyncExternalStore` for outside stores, state that neither copies props nor contradicts itself, and Server Functions and Server Components that guard what crosses to the client. |
766
+ | [`security`](./presets/security/) | 4 | any project | Secrets kept out of logs, error messages, and responses; parameterized SQL; no untrusted input in shell commands, `eval`, or file paths; and secrets compared in constant time. From OWASP's cheat sheets. |
767
+ | [`effect`](./presets/effect/) | 18 | code written with Effect | Steps sequenced with `Effect.gen`, instrumentation attached with `.pipe`, config read through a service and validated, secrets redacted, branded primitives and tagged unions, defects kept apart from typed errors, services built by layers, and tests with their own layers and `TestClock`. |
768
+ | [`alchemy`](./presets/alchemy/) | 41 | code that deploys with [alchemy](https://alchemy.run) | Where Config and bindings are read, which resources keep their data, how secrets stay out of bundles and logs, authorization on public URLs, migrations, durable workflows, and custom providers. |
769
+
770
+ - Name presets in the config's `presets`, or on the command line with
771
+ `--preset`, repeated or separated by commas, as in `--preset effect,alchemy`,
772
+ in which case the config file is optional. Presets named in both places apply
773
+ together.
774
+ - A topic, one of a preset's subdirectories, is a preset of its own:
775
+ `--preset effect/basics` applies only the rules under
776
+ `presets/effect/basics/`, and `alchemy/secrets` only alchemy's secrets rules.
777
+ `security` has no topics.
778
+ - To change a preset's rule without copying it, use [Overrides](#overrides).
779
+
780
+ <details>
781
+ <summary>What every preset does</summary>
782
+
783
+ - It leaves out conventions a linter checks exactly. The notes on each preset
784
+ say which linter checks them.
785
+ - A preset rule says "must" only where its source makes a requirement, and
786
+ "should" where the source gives advice.
787
+ - A topic's rules keep the ids they have in the whole preset, so a topic and its
788
+ preset share cached judgments, and naming both applies each rule once.
789
+ - Where effect or alchemy had a rule that `typescript` or `security` has, that
790
+ one is kept and the other is gone. typescript's
791
+ `async/network-calls-have-timeouts` replaced effect's
792
+ `basics/external-calls-are-resilient`, and `security/parameterized-queries`
793
+ and `security/no-secrets-in-output` replaced alchemy's
794
+ `data/parameterized-sql` and `secrets/never-logged-returned-or-output`. A
795
+ project on Effect or alchemy names `typescript` and `security` too, for those
796
+ rules.
797
+
798
+ </details>
799
+
800
+ <details>
801
+ <summary>Notes on <code>typescript</code></summary>
802
+
803
+ - **Source.** The TypeScript Handbook and the Google TypeScript Style Guide.
804
+ - **Left to a linter.** What typescript-eslint and ESLint check exactly:
805
+ unhandled promises (`no-floating-promises`), throwing non-errors
806
+ (`only-throw-error`), `any` (`no-explicit-any`), exhaustive switches
807
+ (`switch-exhaustiveness-check`), a lost `cause` (`preserve-caught-error`), and
808
+ empty catch blocks (`no-empty`).
809
+ - **Warnings.** Four of its rules report warnings. Three are advice that code
810
+ often has reason to set aside: assertions that claim only what the code
811
+ established, types derived rather than restated, and independent awaits run
812
+ concurrently. The fourth, that HTTP requests carry a timeout, was right on
813
+ fewer than half its findings above 0.8 on the eval: many of the rest were
814
+ calls through a client with a default timeout that the file does not show.
815
+ - **Tests.** `testing/assert-outcomes` and `testing/independent-tests` say
816
+ `tests: only`: they judge tests and nothing else.
817
+ - **Gone.** Two rules the eval found mostly wrong: that tests use fake time
818
+ rather than real waits, and that importing a module does no work.
819
+
820
+ </details>
821
+
822
+ <details>
823
+ <summary>Notes on <code>react</code></summary>
824
+
825
+ - **Source.** react.dev, mostly
826
+ [You Might Not Need an Effect](https://react.dev/learn/you-might-not-need-an-effect)
827
+ and
828
+ [Choosing the State Structure](https://react.dev/learn/choosing-the-state-structure).
829
+ - **Left to a linter.** What the React Compiler's lint rules check, in
830
+ [eslint-plugin-react-hooks](https://react.dev/reference/eslint-plugin-react-hooks)'s
831
+ recommended set and in Oxlint: pure render (`purity`), props and state never
832
+ mutated (`immutability`), refs not read during render (`refs`), no `setState`
833
+ in render or synchronously in an effect (`set-state-in-render`,
834
+ `set-state-in-effect`), which covers state derived in an effect, and no
835
+ component defined inside another (`static-components`).
836
+ - **Files.** Its rules judge `.ts` files as well as `.tsx`, and apply only to
837
+ components, hooks, effects, and Server Functions, so other files cost their
838
+ checks and find nothing.
839
+ - **Warnings.** `server/no-private-data-to-client` reports warnings: a file does
840
+ not show whether the component it passes a record to is a Client Component. So
841
+ does `state/store-ids-not-copies`, which also flags state that copies a string
842
+ or an option from a fixed list, where a copy cannot go stale.
843
+
844
+ </details>
845
+
846
+ <details>
847
+ <summary>Notes on <code>effect</code></summary>
848
+
849
+ - **Source.** The
850
+ [effect-solutions](https://github.com/kitlangton/effect-solutions) docs and
851
+ the [effect/platform](https://effect.website/docs/platform/introduction/)
852
+ docs.
853
+ - **Tests.** Four rules say `tests: only`, so they judge tests and nothing else:
854
+ `testing/test-clock-for-time`, `services/fresh-layer-per-test`,
855
+ `services/test-layers-are-in-memory`, and
856
+ `config/tests-provide-values-directly`.
857
+ - **Guidelines.** Two rules say "should": test layers are in memory, outside
858
+ integration tests, and tests provide config through a layer.
859
+ - **What the rules allow.** Config validation accepts `Config.mapEffect` as well
860
+ as `Config.schema`, and the variants rule does not rule out a `switch`.
861
+ - **Gone.** The rule that a command handler only parses input: the docs show
862
+ that pattern but do not ask for it.
863
+ - **Left to a linter.** Effect's language service,
864
+ [`@effect/tsgo`](https://github.com/Effect-TS/tsgo) on TypeScript 7, checks
865
+ seven conventions exactly, which the preset checked through 0.7. Most are off
866
+ by default. `npx @effect/tsgo setup` installs it, and these lines in its
867
+ plugin options in `tsconfig.json` turn them on:
868
+
869
+ ```jsonc
870
+ "diagnosticSeverity": {
871
+ "strictEffectProvide": "warning", // layers are provided once, at the entry point
872
+ "leakingRequirements": "warning", // service methods have no requirements
873
+ "effectFnOpportunity": "warning", // a named function returning an Effect uses Effect.fn
874
+ "preferSchemaOverJson": "warning", // JSON is decoded with Schema, not JSON.parse
875
+ "nodeBuiltinImport": "warning", // platform services, not node: builtins
876
+ "globalFetch": "warning",
877
+ "globalFetchInEffect": "warning",
878
+ "processEnv": "warning",
879
+ "processEnvInEffect": "warning",
880
+ "globalRandom": "warning", // the Random service, not Math.random
881
+ "globalRandomInEffect": "warning",
882
+ "globalErrorInEffectFailure": "warning", // tagged errors, not Error
883
+ "extendsNativeError": "warning"
884
+ }
885
+ ```
886
+
887
+ `npx @effect/tsgo diagnostics --project tsconfig.json` runs them in CI. Bun
888
+ globals, which the old platform rule also ruled out, need a lint rule of their
889
+ own, such as `no-restricted-globals`.
890
+
891
+ - **What the linter misses.** `leakingRequirements` sees a requirement in an
892
+ operation's type, but not a service built by a factory function that takes its
893
+ dependencies as arguments, whose types have no requirements to find. The
894
+ preset's `services/dependencies-through-layers` asks for that, and
895
+ `services/operations-have-no-requirements` for the style it goes with.
896
+
897
+ </details>
898
+
899
+ <details>
900
+ <summary>Notes on <code>alchemy</code></summary>
901
+
902
+ - **Source.** alchemy's docs and blog.
903
+ - **Warnings.** `providers/idempotent-delete` reports warnings. Whether a delete
904
+ fails on a resource that is already gone is a fact about the API, which the
905
+ file does not show, so the rule also flags deletes of APIs that succeed
906
+ anyway. See [the study](eval/studies/idempotent-delete.md).
907
+
908
+ </details>
909
+
910
+ ### Overrides
531
911
 
532
- ## Must
912
+ A config's `overrides` changes a rule by the id a report names it with, a
913
+ preset's with the preset first, without copying its text into `rules`:
533
914
 
534
915
  ```ts
535
- const Port = Schema.Int.pipe(
536
- Schema.check(Schema.isBetween({ minimum: 1, maximum: 65535 })),
537
- Schema.brand("Port"),
538
- );
916
+ export default defineConfig({
917
+ presets: ["effect", "alchemy"],
918
+ overrides: {
919
+ "alchemy/providers/idempotent-delete": { level: "warning", threshold: 0.9 },
920
+ "effect/basics/instrument-with-pipe": "off",
921
+ },
922
+ });
539
923
  ```
540
924
 
541
- ## Never
925
+ | Value | Effect |
926
+ | ---------------------- | -------------------- |
927
+ | `"off"` | drops the rule |
928
+ | `"warning"`, `"error"` | sets its level |
929
+ | `{ level, threshold }` | sets either, or both |
542
930
 
543
- ```ts
544
- const port: number = Number(process.env.PORT);
545
- ```
546
- ````
931
+ An id that no preset or project rule has refuses the run, so a typo does not go
932
+ unnoticed. A rule of a preset the run does not name is accepted.
547
933
 
548
- `description` is required, with code under `must`, `never`, or both.
549
- `threshold` is optional, and so is `tests`, for a rule that judges tests, which
550
- rules otherwise skip: `tests: only` for a rule about tests, which judges
551
- nothing else, and `tests: include` for one that holds in tests as well. So is
552
- `level`: `level: warning` for a nit, or for a rule that tends to flag code
553
- wrongly, reports its findings as warnings, which do not fail the run. A rule
554
- inline in a config takes the same `tests` and `level`. A file that fails validation refuses
555
- the run with its path in the message. `loadRules(directory)` from the package
556
- root does the same load for your own tooling.
557
-
558
- Jev reads the code under the same words, and is asked whether the file
559
- diverges from the pattern `must` shows, with `never` as an example of
560
- diverging. Write the description in them too, so the rule and its code agree:
561
- "must" and "never", as above.
562
-
563
- A rule with only code that must never be written suits a rule with no single
564
- correct form to show, such as a hand-rolled retry loop or an error caught and
565
- dropped. Neither goes in the other's place: code that must never be written,
566
- under `must`, reads to Jev as the pattern to follow.
567
-
568
- A rule that is a guideline rather than a requirement says "should" instead:
569
- in its description, and in `## Should` and `## Should not` headings (`should`
570
- and `shouldNot` inline in a config). Jev then reads the code under `should`
571
- and `should_not`. A rule is one or the other, so it cannot mix `must` or
572
- `never` with `should` or `should not`. In a config, `reference` and `avoid`,
573
- the names before 0.7, read as `must` and `never`.
574
-
575
- Nested `.adhere/` directories are also discovered, except under
576
- `node_modules/`, `dist/`, and the other skipped directories (above). A rule in
577
- `packages/api/.adhere/rules/data/brand-ports/RULE.md` has the same id,
578
- `data/brand-ports`, but applies only to files under `packages/api/`. Root
579
- `.adhere/` rules apply project-wide. If a nested rule has the same id as a root
580
- rule, the nearest containing `.adhere/` shadows the less-specific rule for that
581
- subtree; outside that subtree, the root rule still applies. There is no broader
582
- priority system: presets are global, project rules override preset rules with
583
- the same id, and nested project rules override less-specific project rules with
584
- the same id for files in their subtree.
585
-
586
- `.adhere/rules/` is the default rather than `docs/adhere/` because it keeps
587
- everything adhere owns in `.adhere/`: the config and the cache are tool
588
- state, not documentation, and would stay in `.adhere/` anyway. A nested
589
- `.adhere/` also scopes its rules to the directory that contains it (above). The
590
- cost is visibility: a dot directory is hidden from `ls`, and from `rg` without
591
- `--hidden`. A repo that would rather keep its rules with the rest of its
592
- documentation can point `rules` at `docs/adhere/`:
934
+ ### Precedence
593
935
 
594
- ```ts
595
- export default defineConfig({ presets: ["effect"], rules: "./docs/adhere" });
596
- ```
936
+ For the model and the threshold, highest first:
597
937
 
598
- That directory holds its rules as `.adhere/rules/` does, a directory per rule,
599
- and they apply project-wide.
938
+ 1. `--threshold` on the command line
939
+ 2. the config file
940
+ 3. presets, in order (a later preset wins)
941
+ 4. the defaults, `jev-latest` and `0.8`
600
942
 
601
- ### Rules as TypeScript files
943
+ For one rule:
602
944
 
603
- A rule's directory can hold it as a `RULE.ts` instead, which default-exports
604
- `defineRule({...})`, taking the fields a config's inline rule does. What
605
- TypeScript adds is `appendState`, a hook on what Jev reads for the rule:
945
+ - A rule's own `threshold` beats all of the above for that rule.
946
+ - An entry in `overrides` beats the rule's own `threshold` and `level`.
947
+ - A rule in `rules` replaces a preset rule with the same id.
606
948
 
607
- ```ts
608
- // .adhere/rules/data/columns/RULE.ts
609
- import { defineRule } from "@drkmttr/adhere";
949
+ `--sufficiency-threshold` beats the config's `sufficiencyThreshold`, which beats
950
+ the default, `0.6`.
610
951
 
611
- export default defineRule({
612
- description:
613
- "A query must name only columns its table has in `schema`, and never a column it lacks.",
614
- must: 'db.select("id", "email").from("users")',
615
- never: 'db.select("mail").from("users")',
616
- appendState: async (state, file, Bun) => ({
617
- schema: await Bun.file("db/schema.sql").text(),
618
- }),
619
- });
952
+ ## Running lint
953
+
954
+ ### Flags
955
+
956
+ | Flag | What it does |
957
+ | ---------------------------------- | ---------------------------------------------------------------------------------------------- |
958
+ | `--preset <names>` | Add built-in rule sets, or their topics. The config becomes optional. See [Presets](#presets). |
959
+ | `--threshold <0 to 1>` | Replace the config's threshold. Per-rule thresholds still apply. |
960
+ | `--sufficiency-threshold <0 to 1>` | Replace the config's `sufficiencyThreshold`: a finding warns when its `context` is below it. |
961
+ | `--yes`, `-y` | Send the requests without asking first. |
962
+ | `--limit <checks>` | Judge at most that many checks. See [Limiting a run](#limiting-a-run). |
963
+ | `--rpm <requests>` | Send at most that many requests a minute. |
964
+ | `--filter <glob>` | Read only the files a glob matches. |
965
+ | `--deny-warnings` | Fail on warnings as well as errors. |
966
+ | `--log-level <level>` | Log the run's work on stderr. See [Logging a run](#logging-a-run). |
967
+
968
+ ### What gets read
969
+
970
+ - `adhere lint` reads the TypeScript files under the working directory: `.ts`,
971
+ `.tsx`, `.mts`, and `.cts`. It does not read declaration files such as
972
+ `.d.ts`, adhere's own config files, or JavaScript.
973
+ - When the working directory contains `agents/`, `apps/`, or `packages/`, only
974
+ those trees are read.
975
+ - Below the working directory, anything under `node_modules/`, `dist/`,
976
+ `coverage/`, `vendor/`, `e2e/`, `references/`, `.adhere/`, `.agents/`,
977
+ `.claude/`, `.direnv/`, `.alchemy/`, or `.vite/` is skipped. The directories
978
+ above the working directory do not count.
979
+ - `.gitignore` is not consulted.
980
+ - A config's `exclude` lists globs, relative to the working directory, of files
981
+ no rule judges, such as generated code. It leaves them out of every run, as
982
+ `--filter '!<glob>'` leaves them out of one:
983
+
984
+ ```ts
985
+ export default defineConfig({ exclude: ["**/generated/**"] });
986
+ ```
987
+
988
+ **Tests** are judged only by rules about tests, whose front matter says `tests`
989
+ (see [Rules as Markdown files](#rules-as-markdown-files)). On alchemy, 38% of
990
+ the findings from rules not about tests were in test helpers and fixtures. A
991
+ test is:
992
+
993
+ - a `.test` or `.spec` file, such as `a.test.ts` or `Button.spec.tsx`;
994
+ - a fixture or test helper named as one, such as `fixtures.ts`,
995
+ `GitHubHttpFixtures.ts`, `ledger-copy.fixture.ts`, `linear-test-helpers.ts`,
996
+ or `test-utils.ts`;
997
+ - any file under a `test/`, `tests/`, `__tests__/`, `testing/`, `fixtures/`,
998
+ `fixture/`, `__fixtures__/`, or `__mocks__/` directory.
999
+
1000
+ ### Before and during a run
1001
+
1002
+ Before it judges anything, `lint` says on stderr what it found and what judging
1003
+ takes, and asks:
1004
+
1005
+ ```text
1006
+ 200 files and 14 rules: 2800 checks, 1400 cached.
1007
+ Judging the other 1400 takes 200 requests to Jev, plus 1 or more for each file with a finding.
1008
+ Those carry about 1.9 million input tokens: about $0.08 at $0.042 per million, and more for locating findings.
1009
+ ? Send 200 requests to Jev, about $0.08? › (Y/n)
620
1010
  ```
621
1011
 
622
- Right before each request about the rule goes out, `appendState` gets the
623
- request's state, the file it is about, and Bun's API, and what it returns is
624
- spread over the state. The state is what Jev reads beside each question:
625
- `code`, the file's sections as Jev reads them, under their numbers from 1, as
626
- in `{ code: { "1": "import …", "2": "export const …" } }`. `file` is the file's
627
- absolute `path`, and its `contents` as written, comments and all. The
628
- description can name a key the hook adds, in backticks, as the questions name
629
- `code`. Nothing the hook returns is checked, so it can change `code` too, or
630
- replace it: a hook can break its own rule's answers, and adhere does not stop
631
- it. It can be async, and one that throws refuses the run, naming the rule.
632
-
633
- A rule with a hook goes to Jev in requests of its own, to judge a file and to
634
- locate a finding, so only that rule reads what its hook adds, and no other
635
- rule's answer depends on it. Each of those requests sends the file again. The
636
- plan counts them, but not the tokens a hook adds, which are not known until it
637
- runs; a request over Jev's context skips the file, as a file too long does.
638
-
639
- The cache keys a judgment by the file's content and the rule, and the hook is
640
- part of the rule by its source: editing the hook judges the rule again. What
641
- the hook reads outside the file is not, so when `db/schema.sql` changes, a file
642
- already judged is not judged again until it or the rule changes.
643
-
644
- A `RULE.ts` is imported, as a config is, so its code runs on every lint, and
645
- one that default-exports no rule refuses the run. It can import other files by
646
- relative path, such as a helper beside it in its directory, but the executable
647
- resolves no packages besides `@drkmttr/adhere`. `Bun` is typed when
648
- `@types/bun` is installed and `.adhere/tsconfig.json` lists it, as
649
- `"types": ["bun"]`, and is `unknown` otherwise. A config's inline rules can
650
- have `appendState` too.
1012
+ - It asks only with a terminal on stdin and stdout, and never when the cache
1013
+ answers every check. `--yes` sends without asking.
1014
+ - The cost is adhere's estimate of the input tokens times the model's price. Jev
1015
+ charges only for input tokens: $0.042 a million for `jev-latest`, per
1016
+ [TypeSafe AI's models page](https://docs.typesafe.ai/models) in
1017
+ September 2026.
1018
+ - A run that stops loses nothing: files finished before it stopped are cached,
1019
+ so running again continues from there.
1020
+
1021
+ <details>
1022
+ <summary>More on unknown prices and blocked files</summary>
1023
+
1024
+ - For a model adhere has no price for, the plan gives the tokens alone.
1025
+ - The firewall in front of Jev's API can refuse a request whose code reads to it
1026
+ as an attack, with a 403 and an HTML page rather than Jev's JSON. It refuses
1027
+ the same request every run, so that file is skipped, the run goes on, and the
1028
+ report lists each such file with Cloudflare's Ray ID, which TypeSafe AI can
1029
+ look the block up by.
1030
+ - When only the question that finds the line is refused, the file's judgments
1031
+ are kept, and a rerun sends that question alone.
1032
+
1033
+ </details>
651
1034
 
652
- ### Scoping a rule
1035
+ ### Limiting a run
653
1036
 
654
- A rule can say where it applies, apart from what it asks for, with
655
- `appliesTo` and `excludeIf`: each a list of descriptions of code, written in
656
- front matter as a JSON array on one line, or as an array inline in a config.
1037
+ Three flags bound what a run does:
657
1038
 
658
- ```md
659
- ---
660
- description: Structured variants must be Schema.TaggedClass members of a Schema.Union, never hand-written object types joined by a tag field.
661
- appliesTo: ["a declared union type or schema"]
662
- excludeIf: ["a type that mirrors a third-party format whose tag key that format fixes, such as a Slack Block Kit block"]
663
- ---
1039
+ | Flag | Effect |
1040
+ | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1041
+ | `--limit <checks>` | Judges at most that many checks, taken in path order. The rest wait, and since judgments are cached, the next run with the same limit picks up where this one stopped. `--limit 0` shows the plan and judges nothing. |
1042
+ | `--rpm <requests>` | Sends at most that many requests to Jev a minute, evenly spaced, retries included. The plan says about how long they take. |
1043
+ | `--filter <glob>` | Reads only the files whose path from the working directory matches, such as `src/**` or `**/*.service.ts`. Wrap a glob in single quotes so the shell does not expand it. Repeat the flag for more, and start a pattern with `!` to leave out what it matches. |
1044
+
1045
+ ```sh
1046
+ adhere lint --filter 'packages/api/**' --filter '!**/generated/**' --limit 200 --rpm 30
664
1047
  ```
665
1048
 
666
- Each goes to Jev as a question of its own, in the same request as the rule,
667
- about the code that breaks the rule rather than the whole file: for
668
- `appliesTo`, whether any of that code is as described, and for `excludeIf`,
669
- whether all of it is. A finding stands only when Jev says yes, above 0.5, to
670
- every `appliesTo` and to no `excludeIf`, so a file with a real violation
671
- beside code an `excludeIf` describes keeps its finding. The scores are cached
672
- with the rule's judgment, and editing a matcher judges the rule again, as
673
- editing its description does. At `--log-level debug`, lint logs each finding
674
- its matchers dropped, with their scores.
1049
+ ### Logging a run
1050
+
1051
+ | Level | What it logs, on stderr |
1052
+ | ------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
1053
+ | `--log-level debug` | The config it loaded, how many paths it listed and how many of them it reads, and each request to Jev, with the file it is for, about how many tokens it carries, the HTTP status, how long it took, and Cloudflare's Ray ID. A failed attempt is logged even when a retry hides it. |
1054
+ | `--log-level trace` | Also each file as it is read, planned, and answered from the cache, and the first 4000 characters of any error Jev's API answers with. |
675
1055
 
676
- Keeping the scope out of the description keeps the description to what the
677
- rule asks for, which the report shows, and gives each part a score of its own.
1056
+ stdout still carries only the report, so the log can go to a file of its own:
678
1057
 
679
- ### Rule writing tips
1058
+ ```sh
1059
+ adhere lint --log-level debug 2> adhere.log
1060
+ ```
680
1061
 
681
- We've evaluated different ways of giving Jev a rule, to catch the most
682
- violations with the fewest false positives. In general:
1062
+ ### Exit codes
683
1063
 
684
- - Say "must" for what code must do and "never" for what it must not, in the
685
- description and in the headings over the code.
686
- - Give one example of each: one `must` block and one `never` block. Three of
687
- each did no better, and several of one kind alone did worse.
688
- - For a guideline rather than a requirement, say "should" and "should not"
689
- the same way.
1064
+ | Code | When |
1065
+ | ---- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
1066
+ | 0 | Nothing is reported, or only warnings. |
1067
+ | 1 | An error is reported. With `--deny-warnings`, a warning too. |
1068
+ | 1 | The run refuses, for example on an invalid config or rule file, a missing API key, or an unknown command or flag. A refusal prints its reason. |
690
1069
 
691
- On the Effect preset, rules written this way cut the findings Jev got wrong at
692
- the default threshold from 6 to 1, and caught as many violations. For more
693
- about the evaluations, see [the eval](eval/README.md). `adhere validate` lists
694
- each rule worded otherwise.
1070
+ ## Comments and suppressions
695
1071
 
696
- ### Presets
1072
+ ### Comments
697
1073
 
698
- A preset has the shape of a config without `presets`: `rules`, and optionally
699
- `model` and `threshold`. Name presets in the config's `presets`, or on the
700
- command line with `--preset`, repeated or separated by commas, as in
701
- `--preset effect,alchemy`, in which case the config file is optional. Presets
702
- named in both places apply together. Reports name a preset's rule with the
703
- preset first, as in `alchemy/secrets/…`. There are five:
704
-
705
- - `typescript`: 11 rules in [`presets/typescript/`](./presets/typescript/) for
706
- any TypeScript project, drawn from the TypeScript Handbook and the Google
707
- TypeScript Style Guide: data from outside checked at runtime, invalid states
708
- unrepresentable, errors never swallowed, resources released on every path,
709
- arguments not mutated, and tests that assert outcomes and stand alone.
710
- - `react`: 10 rules in [`presets/react/`](./presets/react/) for components
711
- and hooks, lifted from react.dev, mostly
712
- [You Might Not Need an Effect](https://react.dev/learn/you-might-not-need-an-effect)
713
- and [Choosing the State Structure](https://react.dev/learn/choosing-the-state-structure):
714
- logic for an event in its handler rather than an effect, effects that clean
715
- up and ignore stale fetches, `useSyncExternalStore` for outside stores, state
716
- that neither copies props nor contradicts itself, and Server Functions and
717
- Server Components that guard what crosses to the client.
718
- - `security`: 4 rules in [`presets/security/`](./presets/security/), drawn
719
- from OWASP's cheat sheets: secrets kept out of logs, error messages, and
720
- responses; parameterized SQL; no untrusted input in shell commands, `eval`,
721
- or file paths; and secrets compared in constant time. It has no topics.
722
- - `effect`: Markdown rules in [`presets/effect/`](./presets/effect/), lifted
723
- from the [effect-solutions](https://github.com/kitlangton/effect-solutions)
724
- docs and the
725
- [effect/platform](https://effect.website/docs/platform/introduction/) docs.
726
- - `alchemy`: 41 rules in [`presets/alchemy/`](./presets/alchemy/) for code that
727
- deploys with [alchemy](https://alchemy.run), lifted from its docs and blog:
728
- where Config and bindings are read, which resources keep their data, how
729
- secrets stay out of bundles and logs, authorization on public URLs,
730
- migrations, durable workflows, and custom providers.
731
-
732
- Each preset divides into topics, its subdirectories, and a topic is a preset
733
- of its own: `--preset effect/basics` applies only the rules under
734
- `presets/effect/basics/`, and `alchemy/secrets` only alchemy's secrets rules.
735
- A topic's rules keep the ids they have in the whole preset, so a topic and its
736
- preset share cached judgments, and naming both applies each rule once.
1074
+ Jev reads a file's code, not its comments. adhere takes every comment out before
1075
+ it hashes or sends a file, blanking it so every line keeps its number. The
1076
+ report's excerpts still show them.
737
1077
 
738
- A config's `overrides` changes a rule by the id a report names it with, a
739
- preset's with the preset first, without copying its text into `rules`: `off`
740
- drops it, `warning` or `error` sets its level, and `{ level, threshold }` sets
741
- either. An id that no preset or project rule has refuses the run, so a typo
742
- does not go unnoticed; a rule of a preset the run does not name is accepted.
1078
+ Jev takes what comments claim at their word, which hid real violations in the
1079
+ studies: see [Writing good rules](#writing-good-rules).
1080
+
1081
+ A comment that says `@adhere` anywhere in it is a note for Jev, and stays. Write
1082
+ one for a fact the code relies on that the file cannot show, once you have
1083
+ checked it, with how you know:
743
1084
 
744
1085
  ```ts
745
- export default defineConfig({
746
- presets: ["effect", "alchemy"],
747
- overrides: {
748
- "alchemy/providers/idempotent-delete": { level: "warning", threshold: 0.9 },
749
- "effect/basics/instrument-with-pipe": "off",
750
- },
751
- });
1086
+ /**
1087
+ * Deletes the activity. @adhere DeleteActivity succeeds on a missing
1088
+ * activity, so no not-found error needs catching; probed 2026-09-25.
1089
+ */
752
1090
  ```
753
1091
 
754
- `effect`'s four rules about tests, `testing/test-clock-for-time`,
755
- `services/fresh-layer-per-test`, `services/test-layers-are-in-memory`, and
756
- `config/tests-provide-values-directly`, say `tests: only`: they judge tests and
757
- nothing else, and every other preset rule skips tests.
758
-
759
- `alchemy`'s `providers/idempotent-delete` reports warnings. Whether a delete
760
- fails on a resource that is already gone is a fact about the API, which the
761
- file does not show, so the rule also flags deletes of APIs that succeed
762
- anyway; see [the study](eval/studies/idempotent-delete.md).
763
-
764
- A preset rule says "must" only where its source makes a requirement, and
765
- "should" where the source gives advice. In `effect`, two rules are
766
- guidelines: test layers are in memory, outside integration tests, and tests
767
- provide config through a layer. Config validation accepts
768
- `Config.mapEffect` as well as `Config.schema`, and the variants rule does not
769
- rule out a `switch`. The rule that a command handler only parses input is
770
- gone: the docs show that pattern but do not ask for it.
771
-
772
- The preset leaves out conventions a linter checks exactly. Effect's language
773
- service, [`@effect/tsgo`](https://github.com/Effect-TS/tsgo) on TypeScript 7,
774
- checks these seven, which the preset checked through 0.7. Most are off by
775
- default; `npx @effect/tsgo setup` installs it, and these lines in its plugin
776
- options in `tsconfig.json` turn them on:
777
-
778
- ```jsonc
779
- "diagnosticSeverity": {
780
- "strictEffectProvide": "warning", // layers are provided once, at the entry point
781
- "leakingRequirements": "warning", // service methods have no requirements
782
- "effectFnOpportunity": "warning", // a named function returning an Effect uses Effect.fn
783
- "preferSchemaOverJson": "warning", // JSON is decoded with Schema, not JSON.parse
784
- "nodeBuiltinImport": "warning", // platform services, not node: builtins
785
- "globalFetch": "warning",
786
- "globalFetchInEffect": "warning",
787
- "processEnv": "warning",
788
- "processEnvInEffect": "warning",
789
- "globalRandom": "warning", // the Random service, not Math.random
790
- "globalRandomInEffect": "warning",
791
- "globalErrorInEffectFailure": "warning", // tagged errors, not Error
792
- "extendsNativeError": "warning"
793
- }
794
- ```
1092
+ - A note informs Jev. It suppresses nothing, and Jev still judges the code
1093
+ around it.
1094
+ - `includeComments: true` in the config sends every comment, for a project whose
1095
+ rules are about comments, such as doc comments on exports.
795
1096
 
796
- `npx @effect/tsgo diagnostics --project tsconfig.json` runs them in CI. Bun
797
- globals, which the old platform rule also ruled out, need a lint rule of their
798
- own, such as `no-restricted-globals`.
799
-
800
- `leakingRequirements` sees a requirement in an operation's type, but not a
801
- service built by a factory function that takes its dependencies as arguments,
802
- whose types have no requirements to find. The preset's
803
- `services/dependencies-through-layers` asks for that, and
804
- `services/operations-have-no-requirements` for the style it goes with.
805
-
806
- `typescript` leaves out what typescript-eslint and ESLint check exactly:
807
- unhandled promises (`no-floating-promises`), throwing non-errors
808
- (`only-throw-error`), `any` (`no-explicit-any`), exhaustive switches
809
- (`switch-exhaustiveness-check`), a lost `cause` (`preserve-caught-error`), and
810
- empty catch blocks (`no-empty`). Four of its rules report warnings. Three are
811
- advice that code often has reason to set aside: assertions that claim only
812
- what the code established, types derived rather than restated, and
813
- independent awaits run concurrently. The fourth, that HTTP requests carry a
814
- timeout, was right on fewer than half its findings above 0.8 on the eval:
815
- many of the rest were calls through a client with a default timeout that the
816
- file does not show.
817
- Two rules the eval found mostly wrong are gone: that tests use fake time
818
- rather than real waits, and that importing a module does no work.
819
-
820
- Where effect or alchemy had a rule that `typescript` or `security` has, that
821
- one is kept and the other is gone: typescript's
822
- `async/network-calls-have-timeouts` replaced effect's
823
- `basics/external-calls-are-resilient`, and `security/parameterized-queries`
824
- and `security/no-secrets-in-output` replaced alchemy's
825
- `data/parameterized-sql` and `secrets/never-logged-returned-or-output`. A
826
- project on Effect or alchemy names those presets too, for those rules.
827
-
828
- `react` leaves out what the React Compiler's lint rules check, in
829
- [eslint-plugin-react-hooks](https://react.dev/reference/eslint-plugin-react-hooks)'s
830
- recommended set and in Oxlint: pure render (`purity`), props and state never
831
- mutated (`immutability`), refs not read during render (`refs`), no `setState`
832
- in render or synchronously in an effect (`set-state-in-render`,
833
- `set-state-in-effect`), which covers state derived in an effect, and no
834
- component defined inside another (`static-components`). Its rules judge `.ts`
835
- files as well as `.tsx`, and apply only to components, hooks, effects, and
836
- Server Functions, so other files cost their checks and find nothing.
837
- `server/no-private-data-to-client` reports warnings: a file does not show
838
- whether the component it passes a record to is a Client Component. So does
839
- `state/store-ids-not-copies`, which also flags state that copies a string or
840
- an option from a fixed list, where a copy cannot go stale.
1097
+ ### Suppressing a finding
841
1098
 
842
- ### Precedence
1099
+ A finding Jev got wrong is suppressed in the code, with a comment saying why:
843
1100
 
844
- Highest first: `--threshold` on the command line, the config file, presets in
845
- order (a later preset wins), then the defaults `jev-latest` and `0.8`. A rule
846
- in `rules` replaces a preset rule with the same id. A rule's own `threshold`
847
- beats all of the above for that rule, and an entry in `overrides` beats the
848
- rule's own `threshold` and `level`. `--sufficiency-threshold` beats the
849
- config's `sufficiencyThreshold`, which beats the default, `0.6`.
1101
+ ```ts
1102
+ // adhere-ignore alchemy/providers/idempotent-delete -- DeleteActivity succeeds on a missing activity; probed 2026-09-25
1103
+ delete: Effect.fn(function* ({ output }) {
1104
+ yield* sfn.deleteActivity({ activityArn: output.activityArn });
1105
+ }),
1106
+ ```
850
1107
 
851
- ## How a file is judged
1108
+ | Comment | What it covers |
1109
+ | --------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------ |
1110
+ | `adhere-ignore`, on a line of its own | The statement that starts on the next line of code: above, the whole handler, wherever in it Jev points, since the lines it points at can move between runs. |
1111
+ | `adhere-ignore`, at the end of a line of code | The statement that starts on that line. |
1112
+ | `adhere-ignore-file`, anywhere in a file | The whole file. The rules it names are not judged there at all. |
852
1113
 
853
- 1. One Jev request per file. The state is the file's sections (see the
854
- report, above) under their numbers from 1, as `{ "1": "…", "2": "…" }`,
855
- without their comments (see [Comments](#comments)), and nothing else. Each rule is one `noul` (yes/no probability) question that
856
- carries the rule's description and its code under its words, `must` and
857
- `never` (or a guideline's `should` and `should_not`): whether the file
858
- diverges from the pattern `must` shows, with `never` as an example of
859
- diverging, or, for a rule with only code that must never be written,
860
- whether the file contains that code. Criteria for each answer draw the line
861
- at the rule's scope, so a file with no code the rule is about is a no. Since
862
- no rule sits in the shared state, a rule's probability depends only on the
863
- file and that rule, not on which other rules share the request. Every
864
- question shares the state cost of the request. On up to 10 files per
865
- project rule, the rule's question has a second one beside it, the linter
866
- check (see [Validate](#validate)), with the same fields. A rule with
867
- `appendState` has requests of its own, here and below, whose state is what
868
- its hook makes of this one (see
869
- [Rules as TypeScript files](#rules-as-typescript-files)).
870
- 2. A second request only when at least one rule's probability is above its
871
- threshold. Per flagged rule, when the file has more than one section, one
872
- `choice` question among the sections' numbers, which yields the section to
873
- report. The options carry no text of their own, since each is a key of the
874
- state: describing each section by its first line, or by what it declares,
875
- located no better on the eval. For each section, two `choice` questions
876
- among its lines, each option the line's text, of the line the violation
877
- starts on and the one it ends on; the chosen section's are underlined.
878
- Beside them, one `noul` carrying the rule:
879
- whether you can tell if `code` breaks it from `code` alone, without knowing
880
- what other files, libraries, services, or configuration do. It yields the
881
- warning (see the report, above).
882
- 3. Jev reads at most 32k tokens of state and one question together, and 64k
883
- tokens in a request. adhere estimates tokens from the JSON it sends, at
884
- about three bytes a token. When a file's questions would not fit in one
885
- request, they are split across several. A file whose code and longest
886
- question would not fit together, or that has more than 255 sections, is
887
- skipped and counted in the summary, and the rest of the run goes on. So is
888
- a file Jev itself counts as over its context, which text denser than the
889
- estimate, such as CJK, can cause.
1114
+ - A comment names rules as a report does, separated by commas.
1115
+ - What follows `--` is the reason, for whoever reads the code next.
1116
+ - A finding without lines is covered when its section overlaps the statement.
1117
+ - Jev never reads these comments, even with `includeComments: true`.
1118
+ - The summary counts the findings they suppress.
890
1119
 
891
1120
  ## Cache
892
1121
 
893
- Judgments are cached in `.adhere/cache/`. Commit it. Every judgment is a paid
894
- request, and Jev's answers vary a little from run to run, so a committed cache
895
- gives everyone and CI the same findings without paying for them again, and a
896
- pull request changes the judgments of only the files it changes.
897
-
898
- `files/` holds Jev's answers by the content they are about: each cache file is
899
- named for the hash of a source file's content as Jev reads it, without its
900
- comments, so a moved or copied file keeps its judgments, identical files share
901
- them, and a change to comments alone re-judges nothing. Each answer sits under a
902
- fingerprint of the model and the question asked for the rule, which carries
903
- the rule's text, and of the source of the rule's `appendState`, if it has one. A changed file re-judges every rule for that file. An edited
904
- rule re-judges only that rule, and an adhere that asks its questions
905
- differently re-judges every rule once. A lowered threshold locates cached
906
- judgments that are newly above it without judging again. `tallies/` keeps the
907
- linter check's answers, one tally per rule, under a key that changes with the
908
- rule's text and the model.
909
-
910
- A cache file is written once and never changed: it is also named for the hash
911
- of its own text. Two branches that judge the same code add files rather than
912
- edit them, so git merges the cache without a conflict. After a run that reads
913
- every file, adhere prunes the cache: it deletes answers about content no file
914
- has, and folds what is left into one file per content. Answers to rules the
915
- run left out stay, so a run with other presets or rules, or with one topic,
916
- loses nothing another run still asks; answers to a rule's old texts stay too,
917
- until the content they are about is gone. Answers about a file the run read
918
- but judged against no rule, such as a test when no rule judges tests, stay as
919
- well, while a file the config excludes is not read, so answers about it go. A
920
- run narrowed by `--filter` does not prune.
1122
+ Judgments are cached in `.adhere/cache/`. **Commit it.** Every judgment is a
1123
+ paid request, and Jev's answers vary a little from run to run, so a committed
1124
+ cache gives everyone and CI the same findings without paying for them again, and
1125
+ a pull request changes the judgments of only the files it changes.
1126
+
1127
+ | What changed | What is judged again |
1128
+ | ------------------------------------------------------------ | ----------------------------------------------------- |
1129
+ | A file's code | Every rule, for that file. |
1130
+ | A file's comments alone | Nothing. |
1131
+ | A file is moved or copied | Nothing. Identical files share their judgments. |
1132
+ | A rule's text, a matcher, or the source of its `appendState` | That rule, for every file. |
1133
+ | What a rule `reads`, such as the workspace's packages | That rule, for every file. |
1134
+ | The threshold is lowered | Nothing. Cached judgments newly above it are located. |
1135
+ | adhere asks its questions differently, after an upgrade | Every rule, once. |
921
1136
 
922
1137
  To keep the cache out of diffs, mark it as generated in `.gitattributes`:
923
1138
 
@@ -925,39 +1140,52 @@ To keep the cache out of diffs, mark it as generated in `.gitattributes`:
925
1140
  .adhere/cache/** linguist-generated -diff
926
1141
  ```
927
1142
 
928
- Anyone who can commit can also write a cache file saying that code passes.
929
- Where lint gates a merge, lint a pull request against the cache as merged,
930
- not as the pull request has it; only the files it changes are judged again:
931
-
932
- ```sh
933
- rm -rf .adhere/cache && git checkout origin/main -- .adhere/cache
934
- adhere lint --yes
935
- ```
936
-
937
- The API key is read only when a request is about to be sent. A run where
938
- every file is cached needs no key and no network.
939
-
940
- ## Agent skill
1143
+ > **Where lint gates a merge.** Anyone who can commit can also write a cache
1144
+ > file saying that code passes. Lint a pull request against the cache as merged,
1145
+ > not as the pull request has it. Only the files it changes are judged again:
1146
+ >
1147
+ > ```sh
1148
+ > rm -rf .adhere/cache && git checkout origin/main -- .adhere/cache
1149
+ > adhere lint --yes
1150
+ > ```
1151
+
1152
+ <details>
1153
+ <summary>How the cache is stored and pruned</summary>
1154
+
1155
+ - `files/` holds Jev's answers by the content they are about. Each cache file is
1156
+ named for the hash of a source file's content as Jev reads it, without its
1157
+ comments.
1158
+ - Each answer sits under a fingerprint of the model and everything the rule
1159
+ sends: its text, its matchers, what it reads, and its `appendState`'s source.
1160
+ - `tallies/` keeps the linter check's answers, one tally per rule, under a key
1161
+ that changes with the rule's text and the model.
1162
+ - A cache file is written once and never changed: it is also named for the hash
1163
+ of its own text. Two branches that judge the same code add files rather than
1164
+ edit them, so git merges the cache without a conflict.
1165
+ - After a run that reads every file, adhere prunes the cache: it deletes answers
1166
+ about content no file has, and folds what is left into one file per content. A
1167
+ run narrowed by `--filter` does not prune.
1168
+ - Pruning keeps answers to rules the run left out, so a run with other presets
1169
+ or rules, or with one topic, loses nothing another run still asks. It keeps
1170
+ answers to a rule's old texts too, until the content they are about is gone.
1171
+ - It also keeps answers about a file the run read but judged against no rule,
1172
+ such as a test when no rule judges tests. A file the config excludes is not
1173
+ read, so answers about it go.
1174
+
1175
+ </details>
1176
+
1177
+ ## Agent skills
941
1178
 
942
1179
  Three skills teach an agent to work with adhere:
943
1180
 
944
- - [`skills/adhere/SKILL.md`](./skills/adhere/SKILL.md) is the reference: the
945
- commands, how to write and word a rule, the config, presets, shared rules,
946
- comments and suppressions, the cache, reading a report, and tuning a rule.
947
- - [`skills/adhere-setup/SKILL.md`](./skills/adhere-setup/SKILL.md) sets adhere
948
- up in a repo with the user. It drafts rules from the repo's conventions,
949
- offers fitting presets and a shared org repo's rules, shows them all in one
950
- list to choose from, imports and validates the chosen ones, walks through
951
- the first lint, and adds a CI workflow.
952
- - [`skills/adhere-fix/SKILL.md`](./skills/adhere-fix/SKILL.md) verifies and
953
- fixes the findings `lint` reports: it checks each against its rule and the
954
- code, fixes the real ones, reports the false ones with the evidence, and says
955
- when a rule is wrong more often than right.
1181
+ | Skill | Print it with | What it does |
1182
+ | ------------------------------------------------ | -------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1183
+ | [`adhere`](./skills/adhere/SKILL.md) | `adhere skill` | The reference: the commands, how to write and word a rule, the config, presets, shared rules, comments and suppressions, the cache, reading a report, and tuning a rule. |
1184
+ | [`adhere-setup`](./skills/adhere-setup/SKILL.md) | `adhere skill setup` | Sets adhere up in a repo with you. See [Generate rules from your repo](#generate-rules-from-your-repo). |
1185
+ | [`adhere-fix`](./skills/adhere-fix/SKILL.md) | `adhere skill fix` | Verifies and fixes the findings `lint` reports: it checks each against its rule and the code, fixes the real ones, reports the false ones with the evidence, and says when a rule is wrong more often than right. |
956
1186
 
957
1187
  Install them with the [skills](https://github.com/vercel-labs/skills) CLI:
958
- `skills add darkmatter/adhere`. The binary carries all three: `adhere skill`
959
- prints the reference, `adhere skill setup` the setup skill, and
960
- `adhere skill fix` the fix skill, so
1188
+ `skills add darkmatter/adhere`. The binary carries all three, so
961
1189
  `adhere skill fix > .agents/skills/adhere-fix/SKILL.md` works without a
962
1190
  checkout.
963
1191
 
@@ -965,8 +1193,7 @@ checkout.
965
1193
 
966
1194
  Pipe the fix skill into an agent that runs without asking. The skill tells it
967
1195
  that invoking the skill permits every `adhere lint` run and its cost, and
968
- `adhere skill fix` prints the skill of the adhere that runs, so its flags
969
- match:
1196
+ `adhere skill fix` prints the skill of the adhere that runs, so its flags match:
970
1197
 
971
1198
  ```sh
972
1199
  adhere skill fix | codex exec --yolo
@@ -987,14 +1214,100 @@ adhere skill fix | codex exec --sandbox workspace-write \
987
1214
  `ADHERE_ALLOW_SUPPRESSIONS=1` is set in its environment, as in
988
1215
  `adhere skill fix | ADHERE_ALLOW_SUPPRESSIONS=1 codex exec --yolo`. It then
989
1216
  suppresses a finding it shows is false, or that three fixes did not clear,
990
- with the evidence in the comment. Without it, the agent leaves those
991
- findings, and its report ends by naming them and the variable.
992
- - End the job with `adhere lint --yes`. The agent exits 0 whatever it left,
993
- so this step passes or fails the job; it judges only the files the agent
994
- changed, and reads the rest from the cache.
995
- - Do not run it on pull requests from forks. The agent reads their code with
996
- your secrets in reach and the network open, so code written to steer it can
997
- send them out.
1217
+ with the evidence in the comment. Without it, the agent leaves those findings,
1218
+ and its report ends by naming them and the variable.
1219
+ - End the job with `adhere lint --yes`. The agent exits 0 whatever it left, so
1220
+ this step passes or fails the job. It judges only the files the agent changed,
1221
+ and reads the rest from the cache.
1222
+
1223
+ > **Do not run it on pull requests from forks.** The agent reads their code with
1224
+ > your secrets in reach and the network open, so code written to steer it can
1225
+ > send them out.
1226
+
1227
+ ## How a file is judged
1228
+
1229
+ 1. **Split.** adhere splits the file into sections of whole statements, in runs
1230
+ of at most 30 lines.
1231
+ 2. **Judge.** One request per file asks Jev, for each rule, how likely the file
1232
+ is to break it.
1233
+ 3. **Locate.** A second request, only for rules above their threshold, asks
1234
+ which section and lines break the rule, and whether the file shows enough to
1235
+ decide.
1236
+
1237
+ A file too long for Jev's context is skipped and counted in the summary.
1238
+
1239
+ <details>
1240
+ <summary>The steps in detail</summary>
1241
+
1242
+ **Split.** adhere splits each file into sections of whole statements, such as
1243
+ its imports, constants, and functions, packed in order into runs of at most 30
1244
+ lines. A class or function longer than that splits into its methods or the
1245
+ statements of its body. Jev reads the file as these sections.
1246
+
1247
+ **Judge: one request per file.**
1248
+
1249
+ - The state is the file's sections under their numbers from 1, as
1250
+ `{ "1": "…", "2": "…" }`, without their comments (see [Comments](#comments)),
1251
+ and nothing else.
1252
+ - Each rule is one `noul` (yes/no probability) question that carries the rule's
1253
+ description and details, and its code under its words, `must` and `never`, or a guideline's
1254
+ `should` and `should_not`. It asks whether the file diverges from the pattern
1255
+ `must` shows, with `never` as an example of diverging, or, for a rule with
1256
+ only code that must never be written, whether the file contains that code.
1257
+ - Criteria for each answer draw the line at the rule's scope, so a file with no
1258
+ code the rule is about is a no.
1259
+ - No rule sits in the shared state, so a rule's probability depends only on the
1260
+ file and that rule, not on which other rules share the request. Every question
1261
+ shares the state cost of the request.
1262
+ - On up to 10 files per project rule, the rule's question has a second one
1263
+ beside it, the linter check (see
1264
+ [Check for contradictions](#check-for-contradictions)), with the same fields.
1265
+ - A rule with `appendState` or `reads` has requests of its own, here and below,
1266
+ whose state adds to this one.
1267
+
1268
+ **Locate: a second request, only when at least one rule's probability is above
1269
+ its threshold.** Per flagged rule:
1270
+
1271
+ - When the file has more than one section, one `choice` question among the
1272
+ sections' numbers, which yields the section to report. The options carry no
1273
+ text of their own, since each is a key of the state: describing each section
1274
+ by its first line, or by what it declares, located no better on the eval.
1275
+ - For each section, two `choice` questions among its lines, each option the
1276
+ line's text: the line the violation starts on and the one it ends on. The
1277
+ chosen section's are underlined. On the eval, lines asked for in this request
1278
+ were named as well as lines asked for on their own, and they held a violation
1279
+ for 94% of real findings, most often as one line
1280
+ ([study](eval/studies/pinpoint.md)).
1281
+ - One `noul` carrying the rule: whether you can tell if `code` breaks it from
1282
+ `code` alone, without knowing what other files, libraries, services, or
1283
+ configuration do. It yields the `context` score and its warning.
1284
+
1285
+ **Limits.** Jev reads at most 32k tokens of state and one question together, and
1286
+ 64k tokens in a request. adhere estimates tokens from the JSON it sends, at
1287
+ about three bytes a token.
1288
+
1289
+ - When a file's questions would not fit in one request, they are split across
1290
+ several.
1291
+ - A file whose code and longest question would not fit together, or that has
1292
+ more than 255 sections, is skipped and counted in the summary, and the rest of
1293
+ the run goes on.
1294
+ - So is a file Jev itself counts as over its context, which text denser than the
1295
+ estimate, such as CJK, can cause.
1296
+
1297
+ </details>
1298
+
1299
+ ## Upgrading
1300
+
1301
+ Names and layouts from older versions:
1302
+
1303
+ | Version | What it had | What happens now |
1304
+ | ----------------- | ------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1305
+ | before 0.7 | A rule's examples were `reference` and `avoid` in a config, and a fence could be tagged `avoid`. | They still read as `must` and `never`. |
1306
+ | through 0.7 | The `effect` preset checked seven conventions a linter checks exactly. | They are left to `@effect/tsgo`. The notes on `effect` under [Presets](#presets) say how to turn them on. |
1307
+ | before 0.13 | A rule was any `*.md` file in `.adhere/`, as `.adhere/data/brand-ports.md`. | A `*.md` file outside every rule's directory, but a `README.md`, refuses the run. The refusal says where each such file goes to keep its id, so `adhere-ignore` comments and `overrides` still name it. |
1308
+ | before 0.13 | `.adhere/tsconfig.json` included only `config.ts`. | Make its `include` `["**/*.ts"]` to give rules written in TypeScript their types. |
1309
+ | 0.13.1 | The workspace's packages had a key of their own, `includeWorkspacePackages: true`. | adhere no longer reads it, and a rule that still has it gets no packages. Write `reads: ["workspacePackages"]` in its place. |
1310
+ | 0.13.1 and before | Prose around a `RULE.md`'s code was ignored. | Jev reads it after the description, as the rule's details, so a rule with prose is judged again once. Keep prose about the rule: why it holds, or where it does not apply. |
998
1311
 
999
1312
  ## Development
1000
1313
 
@@ -1006,31 +1319,33 @@ bun link # puts `adhere` on PATH
1006
1319
  adhere lint --preset effect
1007
1320
  ```
1008
1321
 
1009
- In a checkout no platform package is installed, so `bin/adhere.js` runs
1010
- `src/main.ts` with Bun instead. `bun run typecheck` runs `tsc` and the linter,
1011
- and `bun run test` runs the tests.
1322
+ - In a checkout no platform package is installed, so `bin/adhere.js` runs
1323
+ `src/main.ts` with Bun instead.
1324
+ - `bun run typecheck` runs `tsc` and the linter.
1325
+ - `bun run test` runs the tests.
1012
1326
 
1013
1327
  ### Native executable
1014
1328
 
1015
- `bun run build` compiles `dist/adhere`, a single binary with Bun and the
1016
- presets inside it, through Bun's
1017
- [`--compile`](https://bun.sh/docs/bundler/executables) with
1018
- `--asset ./presets`. It runs without Bun or `node_modules` on the target
1019
- machine and still loads the repo's `config.ts` and Markdown rules from disk.
1020
- `bun run build:npm` compiles it for every published platform instead, each
1021
- into its package under `dist/npm/`
1022
- ([`scripts/npm-packages.ts`](./scripts/npm-packages.ts)).
1023
-
1024
- `adhere --version` tells the builds apart. A release prints its version, as
1025
- `adhere v0.10.0`. `bun run build` prints `git describe --tags --dirty` where
1026
- it was built, as `adhere v0.10.0-16-g9a41328`: 16 commits after v0.10.0, at
1027
- `9a41328`, ending in `-dirty` when the tree had changes. Run from source, it
1028
- prints the checkout's version marked `(source)`.
1329
+ | Command | What it builds |
1330
+ | ------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
1331
+ | `bun run build` | `dist/adhere`, a single binary with Bun and the presets inside it, through Bun's [`--compile`](https://bun.sh/docs/bundler/executables) with `--asset ./presets`. It runs without Bun or `node_modules` on the target machine and still loads the repo's `config.ts` and Markdown rules from disk. |
1332
+ | `bun run build:npm` | The same for every published platform, each into its package under `dist/npm/`, with a copy for the GitHub release in `dist/release/` ([`scripts/npm-packages.ts`](./scripts/npm-packages.ts)). |
1029
1333
 
1030
1334
  Each platform's executable is its own npm package,
1031
- `@drkmttr/adhere-<platform>-<arch>`, limited by `os` and `cpu`, and an
1032
- optional dependency of `@drkmttr/adhere`, so an install fetches only its own
1033
- machine's. The package's `bin/adhere.js` finds it and runs it with Node.
1335
+ `@drkmttr/adhere-<platform>-<arch>`, limited by `os` and `cpu`, and an optional
1336
+ dependency of `@drkmttr/adhere`, so an install fetches only its own machine's.
1337
+ The package's `bin/adhere.js` finds it and runs it with Node. The same
1338
+ executables are attached to each GitHub release, named for `uname -s` and
1339
+ `uname -m` as in `adhere-Linux-x86_64`, so [installing](#install) one needs no
1340
+ Node.
1341
+
1342
+ `adhere --version` tells the builds apart:
1343
+
1344
+ | Build | What it prints |
1345
+ | --------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
1346
+ | A release | Its version, as `adhere v0.10.0`. |
1347
+ | `bun run build` | `git describe --tags --dirty` where it was built, as `adhere v0.10.0-16-g9a41328`: 16 commits after v0.10.0, at `9a41328`, ending in `-dirty` when the tree had changes. |
1348
+ | From source | The checkout's version, marked `(source)`. |
1034
1349
 
1035
1350
  ### Release
1036
1351
 
@@ -1042,26 +1357,37 @@ git tag v0.3.0
1042
1357
  git push origin v0.3.0
1043
1358
  ```
1044
1359
 
1045
- The tag push starts `.github/workflows/release.yaml`, which checks out `main`,
1046
- derives `0.3.0` from `v0.3.0`, and runs `bun run release -- --ci 0.3.0`.
1047
- Release-it bumps `package.json`, commits `chore: release v0.3.0` back to
1048
- `main`, skips npm publish, and creates the GitHub Release for the existing tag.
1049
-
1050
- Publishing stays in `.github/workflows/publish.yaml`: after release-it
1051
- finishes, `.github/workflows/release.yaml` calls the reusable publish workflow
1052
- directly, avoiding a chained GitHub Release event created by `GITHUB_TOKEN`.
1053
- The publish workflow verifies that `package.json` matches the release tag,
1054
- builds the platform packages, and publishes each of them before
1055
- `@drkmttr/adhere`, which it first lists them in as `optionalDependencies` at
1056
- the same version. Every publish is `npm publish --access public --provenance`.
1057
- No npm token is used, so a new package has to be published once by hand
1058
- before it can be given trusted publishers on npmjs.com. npm checks the
1059
- workflow that started the run, so every package has two: `release.yaml`, which
1060
- calls the publish workflow for tag releases, and `publish.yaml`, for manual
1061
- runs. The publish workflow can be run manually for an already-created release
1062
- tag if a publish needs to be retried; it skips every package whose version is
1063
- already on npm. The main package's `files` list keeps it to `bin/`, `src/`,
1064
- `presets/`, and `skills/`.
1360
+ 1. The tag push starts `.github/workflows/release.yaml`, which checks out
1361
+ `main`, derives `0.3.0` from `v0.3.0`, and runs
1362
+ `bun run release -- --ci 0.3.0`.
1363
+ 2. Release-it bumps `package.json`, commits `chore: release v0.3.0` back to
1364
+ `main`, skips npm publish, and creates the GitHub Release for the existing
1365
+ tag.
1366
+ 3. `release.yaml` then calls the reusable publish workflow,
1367
+ `.github/workflows/publish.yaml`, directly, avoiding a chained GitHub Release
1368
+ event created by `GITHUB_TOKEN`.
1369
+ 4. The publish workflow verifies that `package.json` matches the release tag,
1370
+ builds the platform packages, attaches their executables to the GitHub
1371
+ Release, and publishes each package before `@drkmttr/adhere`, which it first
1372
+ lists them in as `optionalDependencies` at the same version.
1373
+
1374
+ <details>
1375
+ <summary>More on publishing</summary>
1376
+
1377
+ - Every publish is `npm publish --access public --provenance`.
1378
+ - No npm token is used, so a new package has to be published once by hand before
1379
+ it can be given trusted publishers on npmjs.com.
1380
+ - npm checks the workflow that started the run, so every package has two trusted
1381
+ publishers: `release.yaml`, which calls the publish workflow for tag releases,
1382
+ and `publish.yaml`, for manual runs.
1383
+ - The publish workflow can be run manually for an already-created release tag if
1384
+ a publish needs to be retried. It skips every package whose version is already
1385
+ on npm, and replaces the executables on the release with ones it builds from
1386
+ `main`.
1387
+ - The main package's `files` list keeps it to `bin/`, `src/`, `presets/`, and
1388
+ `skills/`.
1389
+
1390
+ </details>
1065
1391
 
1066
1392
  ## License
1067
1393