@lorekit/cli 1.52.0 → 1.52.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -597,7 +597,7 @@ erases your cross-repo home lesson.
597
597
  > sharing comes from the account/token, so remote needs no second location.
598
598
  > Different mechanism, same concept.
599
599
 
600
- ### `lorekit migrate` — relocation / rename tool
600
+ ### `lorekit migrate` — relocate a store, or push one to the hosted store
601
601
 
602
602
  Moved or renamed a local store (e.g. an old `.lore/`)? `migrate` re-writes its
603
603
  entries into the current two-tier layout so lessons are never stranded:
@@ -612,6 +612,80 @@ Dry-run (preview) by default; `--yes` (or `--apply`) applies. Idempotent — a
612
612
  re-run is a no-op. It reads LoreKit's own on-disk format only (it does **not**
613
613
  import persistent-memory's `~/.agent-memory/<bucket>/` format).
614
614
 
615
+ #### `--to remote` — bring your local lessons up
616
+
617
+ Started offline and later connected a token? `--to remote` pushes the whole
618
+ local store to the hosted one in a single command, instead of the agent
619
+ re-writing lessons one at a time in remote mode:
620
+
621
+ ```bash
622
+ lorekit migrate --from .lorekit --to remote # dry-run: the plan, per scope
623
+ lorekit migrate --from .lorekit --to remote --yes # push it
624
+ ```
625
+
626
+ The connection and token come from the same place every other command reads
627
+ them (`.mcp.json` / `.lorekit.json` / `LOREKIT_MCP_URL` + `LOREKIT_TOKEN`), so
628
+ a `deny: remote` constraint still wins. Everything is checked **before** the
629
+ first request rather than mid-push:
630
+
631
+ | Condition | Result |
632
+ |---|---|
633
+ | No usable connection | Fails with the `lorekit install` command to run |
634
+ | Read-only `lk_ro_*` token | Fails — a migration writes |
635
+ | Write-only `lk_wo_*` token | Warns, then pushes without reading: its reads are denied, so the destination is not classified at all and every entry reports as `add` (the hosted upsert is idempotent either way) |
636
+ | Unrecognized token prefix | Warns and proceeds, so a self-hosted or custom token still works |
637
+ | `deny: remote` | Fails, naming the config source that denied it |
638
+
639
+ **What transfers, and what the server owns.** `scope`, `key`, `value`,
640
+ `source_agent` and `trigger` travel as the source states them (`value` is
641
+ trimmed server-side), and the original creation date travels as `created_at` —
642
+ so a migrated lesson keeps the recency its ranking depends on. The rest is not
643
+ verbatim, and every departure is reported per entry, in the dry run as well as
644
+ the apply:
645
+
646
+ | Field | What happens |
647
+ |---|---|
648
+ | `updated` | Re-stamped at the write instant — there is no parameter for it |
649
+ | `seen_count` | Starts at 1 for a key the hosted store has never seen; one it already holds lands at ITS count plus one (the RPC treats a write as a sighting, migration 00059). A local tally of 12 never transfers |
650
+ | `tags` | **Replace** the hosted row's labels, so an untagged local entry clears them |
651
+ | `origin_*` | **Sticky** — the RPC coalesces provenance, so an entry carrying none leaves whatever the hosted row already had |
652
+ | `created` | Honoured only when the lesson is new to the hosted store (the RPC's update clause omits it, so one already there keeps its hosted date); an unusable or future-dated value is dropped for the write instant, and the entry is named |
653
+ | `expires_at` | Converted to the remaining whole `ttl_days`; anything beyond the API's 365-day cap lands shortened |
654
+
655
+ **Archived and expired entries are skipped** and reported as such. Neither can
656
+ be represented: a write against an archived key inserts a second, live row
657
+ beside it rather than reviving it, and any TTL would re-date an expired row
658
+ into the future — so pushing them would resurrect lore you retired.
659
+
660
+ Idempotent, with one nuance: a re-run compares only what a hosted write can
661
+ change, and compares it the way the server stores it — trimmed `value`, only
662
+ the provenance the source carries, and an expiry that still honours the local
663
+ intent (the hosted one is fixed at push time while the local one is measured
664
+ from now, so it is judged on whether at least half the intended life remains —
665
+ matching them instant-for-instant could never converge). Comparing the server-owned fields
666
+ directly would report a change forever and re-push the whole store on every
667
+ run. A remote `unchanged` therefore means "the hosted lesson already says
668
+ this", not "byte-identical row".
669
+
670
+ Rate limits and transient failures are handled rather than surfaced as
671
+ failures. The run paces itself under the hosted 120 req/min limit (and says so
672
+ the first time it has to wait), and retries a `429` on the server's own
673
+ `Retry-After` — plus a 5xx or a dropped connection, which on a push of
674
+ thousands of requests is the likeliest failure and the least worth losing an
675
+ entry to. Any other 4xx is a decision, not a blip, and is never retried.
676
+
677
+ Retrying is bounded, so none of this should be read as "it never fails": an
678
+ entry that exhausts its attempts is reported and the run exits non-zero, and
679
+ five CONSECUTIVE failures of any kind stop the run early with a
680
+ partial-progress report — an outage is not worth grinding a whole store through
681
+ the retry budget for. Every one of those exits is safe to re-run. The memory cap ([5,000 active memories on the
682
+ free plan](../../docs/limits.md)) is returned as a `429` too but is terminal —
683
+ the run stops, reports how many entries landed, and exits non-zero, so you can
684
+ archive or upgrade and re-run to resume.
685
+
686
+ Not in v1: `--org <slug>` org-owned writes (a migration lands as your personal
687
+ lore) and the reverse remote → local direction.
688
+
615
689
  ### The control model — two layers, deny-wins
616
690
 
617
691
  Two config layers decide the mode:
@@ -870,7 +944,7 @@ also returns their headroom against the plan's memory cap.
870
944
  | `--mode <mode>` | Memory mode override for `doctor`: `off` / `local` / `remote` |
871
945
  | `--store <path>` | Local project-tier store directory (default `.lorekit`) |
872
946
  | `--from <path>` | Source store to migrate from (`migrate`) |
873
- | `--to <tier>` | Migration destination tier: `home` / `project` (`migrate`; default routes by scope) |
947
+ | `--to <dest>` | Migration destination: `home` / `project` / `remote` (`migrate`; default routes by scope across the local tiers) |
874
948
  | `--apply` | Apply the migration — alias of `--yes` (`migrate`) |
875
949
  | `-y, --yes` | Non-interactive / apply; never prompt |
876
950
  | `--hooks <mode>` | Lifecycle hooks to wire: `all` / `read-only` / `none` (`install`; `none` removes any already wired) |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@lorekit/cli",
3
- "version": "1.52.0",
3
+ "version": "1.52.2",
4
4
  "description": "Install the LoreKit shared-memory skill and run health checks for the LoreKit MCP server.",
5
5
  "license": "MIT",
6
6
  "repository": {
@@ -8,11 +8,16 @@ description: >
8
8
  a slow procedural tier that promotes a recurring lesson into a host rule),
9
9
  chooses the lesson bucket (tag + key namespace) and scopes, and installs the
10
10
  entrenchment guards that stop a learning loop from reinforcing its own
11
- mistakes. Runtime reading and writing of lessons is the lorekit-memory skill;
12
- this is the authoring counterpart. Use when giving a host durable cross-run
13
- memory or wiring a lessons loop. Triggers on "set up memory for my skill",
14
- "add a self-improvement loop", "give my workflow memory", "make this learn
15
- from its mistakes", "self-improving memory", "/lorekit-setup".
11
+ mistakes. Also covers the non-LLM case: giving a deterministic job (a GitHub
12
+ Actions workflow, a cron script, a release pipeline) durable JSON state
13
+ records so it knows what happened on its last run flaky tests, a benchmark
14
+ baseline, the last deployed SHA in the same store agents read. Runtime
15
+ reading and writing of lessons is the lorekit-memory skill; this is the
16
+ authoring counterpart. Use when giving a host durable cross-run memory or
17
+ wiring a lessons loop. Triggers on "set up memory for my skill", "add a
18
+ self-improvement loop", "give my workflow memory", "make this learn from its
19
+ mistakes", "self-improving memory", "memory in CI", "GitHub Actions state",
20
+ "remember the last CI run", "/lorekit-setup".
16
21
  user-invocable: true
17
22
  argument-hint: '[host-name]'
18
23
  license: MIT
@@ -28,20 +33,25 @@ metadata:
28
33
  - loop
29
34
  - authoring
30
35
  - setup
36
+ - ci
37
+ - github-actions
31
38
  ---
32
39
 
33
40
  # LoreKit Setup
34
41
 
35
- Give a skill, workflow, or agent a **self-improvement loop**: it reads its own
36
- accumulated lessons at the start of every run and hardens the proven ones into
37
- permanent rules so it gets better the more it runs.
42
+ Give a host durable cross-run memory. For a model-driven host that means a
43
+ **self-improvement loop**: it reads its own accumulated lessons at the start of
44
+ every run and hardens the proven ones into permanent rules, so it gets better the
45
+ more it runs. For a deterministic host — a CI job — it means **state records**: it
46
+ reads what was true at the end of its last run instead of rediscovering it.
38
47
 
39
48
  This is the **authoring** counterpart to `lorekit-memory`. `lorekit-memory` does
40
49
  the runtime read/write of individual lessons; `lorekit-setup` wires the durable
41
- loop that calls those primitives on a host's behalf. Both run on the same
42
- LoreKit `memory.*` MCP tools.
50
+ memory that calls those primitives on a host's behalf. Both run on the same
51
+ LoreKit store — over the `memory.*` MCP tools for agents, over the `lorekit` CLI
52
+ or REST for jobs.
43
53
 
44
- ## The two tiers (in one screen)
54
+ ## The two tiers of a lessons loop (in one screen)
45
55
 
46
56
  | Tier | Mechanism | Changes behavior? |
47
57
  | ---- | --------- | ----------------- |
@@ -52,7 +62,20 @@ A recurrence gate connects them: a lesson that recurs (`seen_count >= 3`) or is
52
62
  marked `status=structural` becomes promotion-eligible. Entrenchment guards keep
53
63
  the fast tier from reinforcing its own wrong conclusions.
54
64
 
55
- ## Set up a loop
65
+ ## Pick the shape first
66
+
67
+ Two kinds of host want memory, and they want a different record. Decide which
68
+ before reading further:
69
+
70
+ | The host is… | Wants | Read |
71
+ | ------------ | ----- | ---- |
72
+ | A **model-driven** skill, agent, or workflow that fails in recurring, classifiable ways | Prose **lessons** — advisory, recurrence-gated, promotable into rules | [rules/self-improvement-loops.md](./rules/self-improvement-loops.md) |
73
+ | A **deterministic job** — a GitHub Actions workflow, a cron script, a release pipeline — that needs last-run state | JSON **state records** — authoritative, parsed, one key per fact | [rules/ci-state-records.md](./rules/ci-state-records.md) |
74
+
75
+ A host can want both, in separate buckets: the state record carries *what is true
76
+ right now*, the lesson carries *what we learned about it*.
77
+
78
+ ## Set up a loop (model-driven hosts)
56
79
 
57
80
  Follow [rules/self-improvement-loops.md](./rules/self-improvement-loops.md).
58
81
  It covers: when to add a loop (and when not to), the bucket convention (tag
@@ -60,9 +83,21 @@ It covers: when to add a loop (and when not to), the bucket convention (tag
60
83
  read/write steps, the promotion gate, the entrenchment guards, a wiring
61
84
  checklist, and an interactive setup flow.
62
85
 
63
- If invoked with a `host-name`, set up the loop for that host; otherwise ask
64
- which skill / workflow / agent the loop is for, then walk the interactive setup.
86
+ ## Set up CI state (deterministic hosts)
87
+
88
+ Follow [rules/ci-state-records.md](./rules/ci-state-records.md). It covers: when
89
+ LoreKit beats `actions/cache` (and when it does not), the `ci::<job>-state` bucket
90
+ convention, the versioned JSON envelope, the read/write steps with the `lorekit`
91
+ CLI or REST, a full GitHub Actions example, the guards (bounded cardinality, no
92
+ secrets, explicit expiry, never on the critical path, last-write-wins), and a
93
+ wiring checklist.
94
+
95
+ If invoked with a `host-name`, set up memory for that host; otherwise ask which
96
+ skill / workflow / agent / job it is for, pick the shape from the table above,
97
+ then walk that rule file's setup.
65
98
 
66
- The loop's runtime tier needs LoreKit's `memory.*` tools connected; if they are
67
- not, the host's loop is a silent no-op (the slow tier — a normal source edit —
68
- still works). Designing the loop needs no connection.
99
+ A lessons loop's runtime tier needs LoreKit's `memory.*` tools connected; if they
100
+ are not, the host's loop is a silent no-op (the slow tier — a normal source edit —
101
+ still works). A CI job needs a `lk_*` token in its environment instead, and
102
+ degrades to its first-run path when the store is unreachable. Designing either
103
+ needs no connection.
@@ -0,0 +1,452 @@
1
+ # CI state records (programmatic, non-LLM memories)
2
+
3
+ Use this when the host is a **deterministic job** — a GitHub Actions workflow, a
4
+ cron script, a release pipeline — that needs to know **what happened last time**:
5
+ which tests flaked, what the last benchmark number was, which SHA was deployed,
6
+ what it already notified about.
7
+
8
+ This is the same LoreKit store the lessons loop uses, but a different **shape** of
9
+ record. Nothing here is written or interpreted by a model, so the whole apparatus
10
+ of [self-improvement-loops.md](./self-improvement-loops.md) — prose lessons,
11
+ `seen_count` recurrence, promotion, entrenchment guards — does not apply. A
12
+ different, smaller set of guards applies instead.
13
+
14
+ ## Contents
15
+
16
+ - [When this is the right tool (and when it is not)](#when-this-is-the-right-tool-and-when-it-is-not)
17
+ - [State record vs. lesson](#state-record-vs-lesson)
18
+ - [The cardinality rule](#the-cardinality-rule)
19
+ - [Conventions](#conventions)
20
+ - [The record shape](#the-record-shape)
21
+ - [Read step](#read-step)
22
+ - [Write step](#write-step)
23
+ - [TTL is a liveness guard](#ttl-is-a-liveness-guard)
24
+ - [Worked example — GitHub Actions flaky-test tracker](#worked-example--github-actions-flaky-test-tracker)
25
+ - [Guards (do not skip these)](#guards-do-not-skip-these)
26
+ - [The payoff — CI and agents share one store](#the-payoff--ci-and-agents-share-one-store)
27
+ - [Wiring checklist](#wiring-checklist)
28
+
29
+ ---
30
+
31
+ ## When this is the right tool (and when it is not)
32
+
33
+ Reach for a LoreKit state record when **both** hold:
34
+
35
+ - The job needs a **small, current** fact from previous runs — the latest value,
36
+ not a history.
37
+ - That fact is **also useful to an agent or a human**, not just to the next run of
38
+ the same job.
39
+
40
+ Use something else when:
41
+
42
+ | Need | Use instead |
43
+ | ---- | ----------- |
44
+ | Pure job-to-job caching, nobody else reads it | `actions/cache` — free, unlimited, built for this |
45
+ | A full run history / time series | Artifacts, a metrics backend, or your OTel pipeline |
46
+ | Large payloads (coverage reports, logs, traces) | Artifacts — the value cap is 64 KiB |
47
+ | A mutual-exclusion lock between concurrent jobs | A real lock (concurrency groups, an advisory lock) — writes here are last-write-wins with no compare-and-swap |
48
+ | Anything derived from a secret | Nothing. Do not store it. |
49
+
50
+ The second bullet of the "both" test is what earns LoreKit over `actions/cache`.
51
+ If only the next CI run will ever read it, `actions/cache` is the better answer and
52
+ this rule should say so out loud rather than sell the store.
53
+
54
+ ---
55
+
56
+ ## State record vs. lesson
57
+
58
+ | | Lesson (`self-improvement-loops.md`) | CI state record (this file) |
59
+ | --- | --- | --- |
60
+ | Author | a model, at the end of a run | a script, deterministically |
61
+ | Value | prose + a `meta:` comment | JSON (an object, not a bare scalar) |
62
+ | How a reader uses it | **advisory** — a consideration that can be overridden | **authoritative** — parsed and branched on |
63
+ | Recurrence / promotion | yes (`seen_count`, human-gated hardening) | n/a — nothing is inferred, so nothing needs gating |
64
+ | Entrenchment risk | high — the reason those guards exist | none; the risks are cardinality and secrets instead |
65
+ | Key count over time | grows with distinct lessons | **fixed** — see below |
66
+ | Token | the agent's `lk_rw_*` | the job's `lk_wo_*` to write, `lk_ro_*` to read |
67
+
68
+ Because a state record is parsed rather than read, it must be **valid JSON on
69
+ every write** and **version-stamped**, so a reader written against v1 can detect a
70
+ v2 record instead of silently mis-parsing it.
71
+
72
+ ---
73
+
74
+ ## The cardinality rule
75
+
76
+ **One key per fact, overwritten in place. Never one key per run.**
77
+
78
+ This is the single constraint that keeps the idea sound, and it is easy to get
79
+ wrong — "record the state of the last run" reads like an append. It is not.
80
+
81
+ Same `scope` + `key` is an UPDATE, so a job that writes
82
+ `ci-state::flaky-tests` on every run holds **one** row forever, no matter how many
83
+ times it runs. A job that writes `ci-state::run-${{ github.run_id }}` adds a row
84
+ per run and will, in order: crowd the agent context window, blow the 5 000-memory
85
+ cap, and turn a memory store into a bad artifact bucket.
86
+
87
+ Three concrete limits make this a hard rule rather than a style preference:
88
+
89
+ - **64 KiB** per value (`MAX_VALUE_BYTES`) — a 400 above it.
90
+ - **5 000** active memories per user by default, enforced by a DB trigger.
91
+ - **120 requests/min** per user across every LoreKit surface.
92
+
93
+ And one soft limit that bites sooner: the agent SessionStart hook lists each scope
94
+ with a **read cap and no tag filter**, ordered by recency. Per-run CI writes are
95
+ the most recently updated rows in the repo scope, so they would displace the
96
+ lessons the hook exists to inject. Bounded cardinality is what keeps CI records
97
+ cheap enough to live in the same scope agents read.
98
+
99
+ ---
100
+
101
+ ## Conventions
102
+
103
+ Give the state its own bucket, in a namespace that can never be mistaken for a
104
+ lesson bucket:
105
+
106
+ - **Tag:** `ci::<job>-state` — e.g. `ci::test-state`, `ci::deploy-state`.
107
+ Deliberately **not** `loop::…`; that prefix is the lessons grammar.
108
+ - **Key:** `ci-state::<slug>` — e.g. `ci-state::flaky-tests`. One slug per fact.
109
+ - **Taxonomy:** pass `--kind bus --host ci` explicitly. `kind`/`host` are only
110
+ inferred from `loop::` tags, so a `ci::` tag leaves them NULL unless you say so.
111
+ Setting them buys `lorekit list --kind bus --host ci` — one command that shows
112
+ every state record and nothing else.
113
+
114
+ Scope, by what the fact is about:
115
+
116
+ - **`repo::{owner}/{repo}`** — the default. Trunk state: flaky tests, the last
117
+ deployed SHA, a benchmark baseline.
118
+ - **`branch::{owner}/{repo}::{branch}`** — per-PR state that should disappear with
119
+ the branch (what this PR's last run already commented on). Pair it with
120
+ `--ttl-days` so it self-cleans.
121
+ - **`global`** — almost never. A CI fact is repo-bound by construction.
122
+
123
+ ---
124
+
125
+ ## The record shape
126
+
127
+ A JSON object with a version stamp and a provenance block, so a reader can tell
128
+ which run produced it and whether it understands the format:
129
+
130
+ ```json
131
+ {
132
+ "v": 1,
133
+ "updated_by_run": "https://github.com/owner/repo/actions/runs/123456789",
134
+ "commit": "0f4a1c9…",
135
+ "data": {
136
+ "flaky": ["src/queue.test.ts::retries on 429", "src/auth.test.ts::refresh"],
137
+ "consecutive_green": 3
138
+ }
139
+ }
140
+ ```
141
+
142
+ Rules that make it safe to parse:
143
+
144
+ - `v` is required and bumped on any breaking shape change. A reader that sees an
145
+ unknown `v` **falls back to its first-run path and logs it** — it never guesses.
146
+ - Everything mutable lives under `data`, so the envelope stays stable.
147
+ - No secrets, no tokens, no full environment dumps, no raw log bodies (see
148
+ [Guards](#guards-do-not-skip-these)).
149
+
150
+ ---
151
+
152
+ ## Read step
153
+
154
+ Address the record with **`--scope` and `--key` flags, not the single-token
155
+ `<scope::key>` form** — the key itself contains `::`, and flags are the only
156
+ unambiguous way to express that.
157
+
158
+ ```bash
159
+ set -euo pipefail
160
+
161
+ STATE_JSON='{}'
162
+ if lorekit show --scope "repo::${REPO}" --key 'ci-state::flaky-tests' \
163
+ --remote --json > state-raw.json 2>&1; then
164
+ cat state-raw.json # the log-visibility rule: always echo
165
+ STATE_JSON=$(jq -r '.remote.record.value // "{}"' state-raw.json)
166
+ else
167
+ cat state-raw.json
168
+ echo "No prior state (first run, or LoreKit unreachable) — continuing with defaults."
169
+ fi
170
+
171
+ VERSION=$(jq -r '.v // 0' <<<"$STATE_JSON")
172
+ if [ "$VERSION" != "1" ]; then
173
+ echo "State schema v${VERSION} is not v1 — ignoring it and rebuilding from scratch."
174
+ STATE_JSON='{}'
175
+ fi
176
+ ```
177
+
178
+ Two things this deliberately gets right:
179
+
180
+ - **`lorekit show` exits 1 on a miss.** That is not an error condition here — the
181
+ first run of any new state record misses. Branch on it; do not `|| true` it away,
182
+ which would swallow a genuine auth or network failure too.
183
+ - **A LoreKit outage degrades to the first-run path.** The job continues; it does
184
+ not fail. See [Guards](#guards-do-not-skip-these).
185
+
186
+ ---
187
+
188
+ ## Write step
189
+
190
+ ```bash
191
+ set -euo pipefail
192
+
193
+ jq -nc \
194
+ --arg run "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" \
195
+ --arg sha "${GITHUB_SHA}" \
196
+ --argjson data "$NEW_DATA" \
197
+ '{v: 1, updated_by_run: $run, commit: $sha, data: $data}' \
198
+ | lorekit write \
199
+ --scope "repo::${REPO}" \
200
+ --key 'ci-state::flaky-tests' \
201
+ --tags 'ci::test-state' \
202
+ --kind bus --host ci \
203
+ --ttl-days 7 \
204
+ --remote --json \
205
+ | tee write-result.json
206
+ ```
207
+
208
+ Notes on each flag that is not obvious:
209
+
210
+ - **`--ttl-days` is not optional, and it should be short.** See
211
+ [TTL is a liveness guard](#ttl-is-a-liveness-guard) — the countdown refreshes on
212
+ every write, so a short TTL expires the record only when the *job* stops running.
213
+ Never omit it: a write passing neither `--ttl-days` nor `--clear-ttl` inherits
214
+ whatever `ttl.default` / `scope.defaults.<prefix>.ttl_days` the repo config
215
+ happens to set for lessons, which is a number nobody chose for this record.
216
+ - **No value argument** — `lorekit write` reads stdin when none is given, which is
217
+ what makes the `jq | lorekit write` pipe work.
218
+ - **`--remote`** — be explicit. A CI checkout has no offline store, and picking the
219
+ target by inference is not something to leave to chance in a pipeline.
220
+ - **`--kind bus --host ci`** — see [Conventions](#conventions).
221
+
222
+ The token comes from `LOREKIT_TOKEN` in the environment. Use a **write-only
223
+ `lk_wo_*` token for the writing job**: it cannot read anything back, so a leaked CI
224
+ token cannot exfiltrate the team's lore. Where the same job must also read, it needs
225
+ `lk_rw_*` — split the steps and use two tokens if the read is small enough to
226
+ justify it.
227
+
228
+ Over REST instead of the CLI, the equivalents are `POST /memories` (write) and
229
+ `GET /memories?scope=…&key=…` (read), same auth header. The CLI is preferred in CI
230
+ because it derives `origin` (repo / branch / commit / PR) from the GitHub Actions
231
+ environment automatically, which is what makes each record traceable back to the
232
+ run that wrote it in the dashboard.
233
+
234
+ ---
235
+
236
+ ## TTL is a liveness guard
237
+
238
+ **Default to a short TTL — roughly a week — not a permanent record.** This is the
239
+ opposite of the instinct that "state must persist", and it follows from one detail
240
+ of the write path.
241
+
242
+ `memory_write` computes `expires_at = now() + ttl_days` on **every** write, insert
243
+ or update (`supabase/migrations/00030_memory_ttl.sql`: `expires_at = case when
244
+ p_ttl_days is not null then v_expires_at else memories.expires_at end`). The job
245
+ rewrites its record on every run and passes `--ttl-days` every time, so the
246
+ countdown restarts every run.
247
+
248
+ That makes the TTL measure **how long the job has been silent**, not how old the
249
+ record is. A record expires when — and only when — the job stopped running for that
250
+ long, which is exactly when its contents stopped being true. A daily job with a
251
+ 7-day TTL keeps its state indefinitely while it runs daily, and drops it a week
252
+ after someone deletes the workflow.
253
+
254
+ | Job cadence | TTL | Why |
255
+ | ----------- | --- | --- |
256
+ | Every push / per-PR | **7 days** | Survives a feature freeze or a quiet holiday week; an abandoned job self-cleans |
257
+ | Nightly | **14 days** | Tolerates a fortnight of red or paused schedules |
258
+ | Weekly (release, audit) | **30 days** | ~4 missed runs of slack |
259
+ | Branch-scoped, any cadence | **7 days or less** | The branch outlives the state; let it decay with the PR |
260
+
261
+ Pick the number so a *normal* quiet spell does not expire the record, and an
262
+ abandoned job does. Do not reach for 365 to be safe — that is just "permanent" with
263
+ extra steps, and it re-creates the failure below.
264
+
265
+ Two things this buys beyond freshness:
266
+
267
+ - **A stale record is worse than a missing one.** Falling back to the first-run path
268
+ is a defined, tested code path. Acting on a flaky-test set from four months ago is
269
+ not — it is silently wrong, and nothing surfaces that.
270
+ - **It bounds the blast radius of a cardinality mistake.** [The cardinality
271
+ rule](#the-cardinality-rule) is a discipline, and disciplines get violated. If
272
+ someone keys on `github.run_id` anyway, a 7-day TTL turns unbounded growth into a
273
+ bounded steady state that drains itself — the store stops filling instead of
274
+ climbing to the 5 000-memory cap.
275
+
276
+ **`--clear-ttl` is the rare exception, not the default.** Reserve it for a record
277
+ whose *absence* is more dangerous than its staleness — a migration watermark, say.
278
+ If you find yourself there, ask first whether the fact belongs in a best-effort
279
+ memory store at all rather than in the datastore that owns it.
280
+
281
+ ---
282
+
283
+ ## Worked example — GitHub Actions flaky-test tracker
284
+
285
+ Applies the usual workflow-authoring rules (the `github-actions-author` skill in
286
+ [mthines/agent-skills](https://github.com/mthines/agent-skills) is the reference):
287
+ named steps, least-privilege `permissions`, SHA-pinned third-party actions,
288
+ `concurrency` without cancellation (a cancelled run must not leave half-written
289
+ state), `set -euo pipefail`, and every command's output reaching the run log.
290
+
291
+ ```yaml
292
+ name: Tests
293
+
294
+ on:
295
+ push:
296
+ branches: [main]
297
+ paths: ['src/**', 'package-lock.json', '.github/workflows/tests.yml']
298
+
299
+ permissions:
300
+ contents: read
301
+
302
+ concurrency:
303
+ group: tests-${{ github.ref }}
304
+ cancel-in-progress: false # never abandon a run mid-state-write
305
+
306
+ jobs:
307
+ test:
308
+ runs-on: ubuntu-latest
309
+ env:
310
+ LOREKIT_TOKEN: ${{ secrets.LOREKIT_TOKEN_RW }}
311
+ REPO: ${{ github.repository }}
312
+ steps:
313
+ - name: Check out the repository
314
+ uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
315
+
316
+ - name: Set up Node.js
317
+ uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
318
+ with:
319
+ node-version: 22
320
+ cache: npm
321
+
322
+ - name: Install dependencies
323
+ run: |
324
+ set -euo pipefail
325
+ npm ci 2>&1 | tee install.log
326
+
327
+ - name: Read the known-flaky set from LoreKit
328
+ run: |
329
+ set -euo pipefail
330
+ KNOWN='[]'
331
+ if npx --yes @lorekit/cli@latest show \
332
+ --scope "repo::${REPO}" --key 'ci-state::flaky-tests' \
333
+ --remote --json > state.json 2>&1; then
334
+ cat state.json
335
+ KNOWN=$(jq -c '(.remote.record.value // "{}") | fromjson
336
+ | if .v == 1 then .data.flaky else [] end' state.json)
337
+ else
338
+ cat state.json
339
+ echo "No prior flaky state — treating every failure as new."
340
+ fi
341
+ echo "KNOWN_FLAKY=${KNOWN}" >> "$GITHUB_ENV"
342
+ echo "Known flaky on entry: ${KNOWN}"
343
+
344
+ - name: Run the test suite
345
+ id: tests
346
+ run: |
347
+ set -euo pipefail
348
+ npm test -- --reporter=default --reporter=json --outputFile=results.json 2>&1 \
349
+ | tee test-output.log
350
+
351
+ - name: Record the flaky set back to LoreKit
352
+ if: always()
353
+ run: |
354
+ set -euo pipefail
355
+ FLAKY=$(jq -c '[.testResults[]? | select(.status == "failed") | .name]' results.json)
356
+ jq -nc \
357
+ --arg run "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" \
358
+ --arg sha "${GITHUB_SHA}" \
359
+ --argjson flaky "$FLAKY" \
360
+ '{v: 1, updated_by_run: $run, commit: $sha, data: {flaky: $flaky}}' \
361
+ | npx --yes @lorekit/cli@latest write \
362
+ --scope "repo::${REPO}" --key 'ci-state::flaky-tests' \
363
+ --tags 'ci::test-state' --kind bus --host ci \
364
+ --ttl-days 7 --remote --json \
365
+ | tee lorekit-write.log \
366
+ || echo "LoreKit write failed (exit $?) — not failing the build; see the output above."
367
+ ```
368
+
369
+ The last step is `if: always()` and its failure is swallowed **with the output
370
+ echoed and the exit code named** — a memory store being down is never a reason to
371
+ turn a green test run red, but a silent `|| true` would hide a misconfigured token
372
+ for months.
373
+
374
+ ---
375
+
376
+ ## Guards (do not skip these)
377
+
378
+ The lessons loop's entrenchment guards do not apply — nothing here is inferred. These
379
+ do:
380
+
381
+ 1. **Bounded cardinality.** One key per fact, overwritten in place. If the number
382
+ of keys a job writes grows with the number of runs, the design is wrong. See
383
+ [The cardinality rule](#the-cardinality-rule).
384
+ 2. **No secrets, ever — and the bar is higher here than for lessons.** A lesson is
385
+ drafted by a model that can be told to redact; a state record is assembled
386
+ mechanically from a CI environment that is *full* of tokens. Never
387
+ `env | jq -R`, never a raw log body, never an error string that might carry a
388
+ connection URL. Build the payload from an explicit allow-list of fields.
389
+ 3. **Expiry is explicit, and short by default.** Pass `--ttl-days` sized to the
390
+ job's cadence (~7 days for anything running at least daily). It refreshes on
391
+ every write, so it expires only when the job goes silent — see
392
+ [TTL is a liveness guard](#ttl-is-a-liveness-guard). Never inherit the config
393
+ default by accident, and reserve `--clear-ttl` for the rare record whose absence
394
+ is worse than its staleness.
395
+ 4. **The store is never on the critical path.** A read miss falls back to the
396
+ first-run path; a write failure is logged and swallowed. A LoreKit outage must
397
+ not fail a build — but it must be *visible* in the log, never `|| true`-d away.
398
+ 5. **Last write wins; there is no compare-and-swap.** Two concurrent jobs writing
399
+ the same key will clobber each other. Either write from one job, or serialise
400
+ with a `concurrency` group, or shard the key.
401
+ 6. **Version every record.** An unrecognised `v` means fall back to the first-run
402
+ path and log it — never parse a shape you do not recognise.
403
+ 7. **Remember agents read this scope.** State records land in `repo::` alongside
404
+ lessons and will surface in an agent's SessionStart injection. That is the point
405
+ (see below) — but it means the value should read sensibly to a human skimming
406
+ it, and stay small.
407
+
408
+ ---
409
+
410
+ ## The payoff — CI and agents share one store
411
+
412
+ Job-to-job persistence alone does not justify LoreKit over `actions/cache`. What
413
+ does is that **both sides of the loop read the same store**:
414
+
415
+ - CI writes `ci-state::flaky-tests` deterministically on every run of `main`.
416
+ - An agent asked to "fix the flaky tests" reads it at SessionStart — the same repo
417
+ scope, no extra wiring — and starts from the real list instead of re-deriving it.
418
+ - The agent's own findings go back as a **lesson** in the `loop::` bucket, in the
419
+ same repo scope, where the next human and the next agent both see it.
420
+ - The dashboard shows both, with each state record linked back to the exact
421
+ workflow run that wrote it via the derived `origin`.
422
+
423
+ So the two record kinds are complementary, not competing: the state record carries
424
+ **what is true right now**, the lesson carries **what we learned about it**. Keep
425
+ them in separate buckets (`ci::` vs `loop::`) so each can be read, filtered, and
426
+ groomed on its own terms.
427
+
428
+ ---
429
+
430
+ ## Wiring checklist
431
+
432
+ To add CI state to a job called `<job>`:
433
+
434
+ - [ ] Confirm the fact is small, current-only, and useful to more than the next CI
435
+ run — otherwise use `actions/cache`.
436
+ - [ ] Pick the bucket: tag `ci::<job>-state`, key `ci-state::<slug>`, one slug per
437
+ fact, `--kind bus --host ci`.
438
+ - [ ] Pick the scope: `repo::` for trunk state, `branch::` for per-PR state.
439
+ - [ ] Pick the TTL from the job's cadence (7 / 14 / 30 days — see the table). Short
440
+ by default; `--clear-ttl` only with a reason.
441
+ - [ ] Define the JSON envelope with a `v` stamp, and the reader's unknown-version
442
+ fallback.
443
+ - [ ] Add the **read step** early in the job (`--scope`/`--key` flags, exit-1-is-a-miss
444
+ branch, output echoed).
445
+ - [ ] Add the **write step** at the end (`if: always()` if the state should survive a
446
+ failing run), `--ttl-days` passed on EVERY write, failure swallowed but logged.
447
+ - [ ] Provision tokens: `lk_wo_*` for a write-only job, `lk_ro_*` for a read-only
448
+ one, `lk_rw_*` only where one job genuinely needs both.
449
+ - [ ] Verify the payload is built from an explicit allow-list of fields, with no
450
+ environment dump and no raw error bodies.
451
+ - [ ] Confirm the key count does not grow with run count — run the job twice and
452
+ check `lorekit list --kind bus --host ci` still shows one row per fact.
@@ -46,6 +46,10 @@ Do **not** add a loop to:
46
46
  must not be biased by prior runs; that is the whole point of a fresh pass.
47
47
  - **Steps that handle secrets or credentials** — routing them through lesson
48
48
  extraction is a leak risk.
49
+ - **Deterministic jobs** (a CI workflow, a cron script, a release pipeline) — they
50
+ do not draft prose lessons or need a promotion gate; what they want is a JSON
51
+ **state record** of their last run. That is a different shape with different
52
+ guards: see [ci-state-records.md](./ci-state-records.md).
49
53
 
50
54
  When unsure, default to **no**. A loop can be added later; unwinding an
51
55
  entrenched-bias loop is harder.