@lorekit/cli 1.52.1 → 1.52.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/package.json
CHANGED
|
@@ -62,4 +62,8 @@ Wildcards work **only** in `memory.search` — not in `memory.list`,
|
|
|
62
62
|
2. `repo::` must include a `/` (owner/repo); `repo::mthines` → 400.
|
|
63
63
|
3. `branch::` must have exactly two `::` separators.
|
|
64
64
|
4. Only `global`, `project`, `repo`, `branch` prefixes are valid.
|
|
65
|
-
5. Segments are trimmed and lowercased on ingest.
|
|
65
|
+
5. Segments are trimmed and lowercased on ingest by the MCP tools (`memory.write`
|
|
66
|
+
validates through the normalising `validateScope`). The REST write path
|
|
67
|
+
(`POST /memories`) stores the scope EXACTLY as sent, so `memories.scope` can
|
|
68
|
+
hold mixed case — which is why a `?scope=` filter on the `/memories` routes is
|
|
69
|
+
validated but never lowercased and matches the stored string exactly.
|
|
@@ -8,11 +8,16 @@ description: >
|
|
|
8
8
|
a slow procedural tier that promotes a recurring lesson into a host rule),
|
|
9
9
|
chooses the lesson bucket (tag + key namespace) and scopes, and installs the
|
|
10
10
|
entrenchment guards that stop a learning loop from reinforcing its own
|
|
11
|
-
mistakes.
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
|
|
11
|
+
mistakes. Also covers the non-LLM case: giving a deterministic job (a GitHub
|
|
12
|
+
Actions workflow, a cron script, a release pipeline) durable JSON state
|
|
13
|
+
records so it knows what happened on its last run — flaky tests, a benchmark
|
|
14
|
+
baseline, the last deployed SHA — in the same store agents read. Runtime
|
|
15
|
+
reading and writing of lessons is the lorekit-memory skill; this is the
|
|
16
|
+
authoring counterpart. Use when giving a host durable cross-run memory or
|
|
17
|
+
wiring a lessons loop. Triggers on "set up memory for my skill", "add a
|
|
18
|
+
self-improvement loop", "give my workflow memory", "make this learn from its
|
|
19
|
+
mistakes", "self-improving memory", "memory in CI", "GitHub Actions state",
|
|
20
|
+
"remember the last CI run", "/lorekit-setup".
|
|
16
21
|
user-invocable: true
|
|
17
22
|
argument-hint: '[host-name]'
|
|
18
23
|
license: MIT
|
|
@@ -28,20 +33,25 @@ metadata:
|
|
|
28
33
|
- loop
|
|
29
34
|
- authoring
|
|
30
35
|
- setup
|
|
36
|
+
- ci
|
|
37
|
+
- github-actions
|
|
31
38
|
---
|
|
32
39
|
|
|
33
40
|
# LoreKit Setup
|
|
34
41
|
|
|
35
|
-
Give a
|
|
36
|
-
|
|
37
|
-
permanent rules
|
|
42
|
+
Give a host durable cross-run memory. For a model-driven host that means a
|
|
43
|
+
**self-improvement loop**: it reads its own accumulated lessons at the start of
|
|
44
|
+
every run and hardens the proven ones into permanent rules, so it gets better the
|
|
45
|
+
more it runs. For a deterministic host — a CI job — it means **state records**: it
|
|
46
|
+
reads what was true at the end of its last run instead of rediscovering it.
|
|
38
47
|
|
|
39
48
|
This is the **authoring** counterpart to `lorekit-memory`. `lorekit-memory` does
|
|
40
49
|
the runtime read/write of individual lessons; `lorekit-setup` wires the durable
|
|
41
|
-
|
|
42
|
-
LoreKit `memory.*` MCP tools
|
|
50
|
+
memory that calls those primitives on a host's behalf. Both run on the same
|
|
51
|
+
LoreKit store — over the `memory.*` MCP tools for agents, over the `lorekit` CLI
|
|
52
|
+
or REST for jobs.
|
|
43
53
|
|
|
44
|
-
## The two tiers (in one screen)
|
|
54
|
+
## The two tiers of a lessons loop (in one screen)
|
|
45
55
|
|
|
46
56
|
| Tier | Mechanism | Changes behavior? |
|
|
47
57
|
| ---- | --------- | ----------------- |
|
|
@@ -52,7 +62,20 @@ A recurrence gate connects them: a lesson that recurs (`seen_count >= 3`) or is
|
|
|
52
62
|
marked `status=structural` becomes promotion-eligible. Entrenchment guards keep
|
|
53
63
|
the fast tier from reinforcing its own wrong conclusions.
|
|
54
64
|
|
|
55
|
-
##
|
|
65
|
+
## Pick the shape first
|
|
66
|
+
|
|
67
|
+
Two kinds of host want memory, and they want a different record. Decide which
|
|
68
|
+
before reading further:
|
|
69
|
+
|
|
70
|
+
| The host is… | Wants | Read |
|
|
71
|
+
| ------------ | ----- | ---- |
|
|
72
|
+
| A **model-driven** skill, agent, or workflow that fails in recurring, classifiable ways | Prose **lessons** — advisory, recurrence-gated, promotable into rules | [rules/self-improvement-loops.md](./rules/self-improvement-loops.md) |
|
|
73
|
+
| A **deterministic job** — a GitHub Actions workflow, a cron script, a release pipeline — that needs last-run state | JSON **state records** — authoritative, parsed, one key per fact | [rules/ci-state-records.md](./rules/ci-state-records.md) |
|
|
74
|
+
|
|
75
|
+
A host can want both, in separate buckets: the state record carries *what is true
|
|
76
|
+
right now*, the lesson carries *what we learned about it*.
|
|
77
|
+
|
|
78
|
+
## Set up a loop (model-driven hosts)
|
|
56
79
|
|
|
57
80
|
Follow [rules/self-improvement-loops.md](./rules/self-improvement-loops.md).
|
|
58
81
|
It covers: when to add a loop (and when not to), the bucket convention (tag
|
|
@@ -60,9 +83,21 @@ It covers: when to add a loop (and when not to), the bucket convention (tag
|
|
|
60
83
|
read/write steps, the promotion gate, the entrenchment guards, a wiring
|
|
61
84
|
checklist, and an interactive setup flow.
|
|
62
85
|
|
|
63
|
-
|
|
64
|
-
|
|
86
|
+
## Set up CI state (deterministic hosts)
|
|
87
|
+
|
|
88
|
+
Follow [rules/ci-state-records.md](./rules/ci-state-records.md). It covers: when
|
|
89
|
+
LoreKit beats `actions/cache` (and when it does not), the `ci::<job>-state` bucket
|
|
90
|
+
convention, the versioned JSON envelope, the read/write steps with the `lorekit`
|
|
91
|
+
CLI or REST, a full GitHub Actions example, the guards (bounded cardinality, no
|
|
92
|
+
secrets, explicit expiry, never on the critical path, last-write-wins), and a
|
|
93
|
+
wiring checklist.
|
|
94
|
+
|
|
95
|
+
If invoked with a `host-name`, set up memory for that host; otherwise ask which
|
|
96
|
+
skill / workflow / agent / job it is for, pick the shape from the table above,
|
|
97
|
+
then walk that rule file's setup.
|
|
65
98
|
|
|
66
|
-
|
|
67
|
-
not, the host's loop is a silent no-op (the slow tier — a normal source edit —
|
|
68
|
-
still works).
|
|
99
|
+
A lessons loop's runtime tier needs LoreKit's `memory.*` tools connected; if they
|
|
100
|
+
are not, the host's loop is a silent no-op (the slow tier — a normal source edit —
|
|
101
|
+
still works). A CI job needs a `lk_*` token in its environment instead, and
|
|
102
|
+
degrades to its first-run path when the store is unreachable. Designing either
|
|
103
|
+
needs no connection.
|
|
@@ -0,0 +1,452 @@
|
|
|
1
|
+
# CI state records (programmatic, non-LLM memories)
|
|
2
|
+
|
|
3
|
+
Use this when the host is a **deterministic job** — a GitHub Actions workflow, a
|
|
4
|
+
cron script, a release pipeline — that needs to know **what happened last time**:
|
|
5
|
+
which tests flaked, what the last benchmark number was, which SHA was deployed,
|
|
6
|
+
what it already notified about.
|
|
7
|
+
|
|
8
|
+
This is the same LoreKit store the lessons loop uses, but a different **shape** of
|
|
9
|
+
record. Nothing here is written or interpreted by a model, so the whole apparatus
|
|
10
|
+
of [self-improvement-loops.md](./self-improvement-loops.md) — prose lessons,
|
|
11
|
+
`seen_count` recurrence, promotion, entrenchment guards — does not apply. A
|
|
12
|
+
different, smaller set of guards applies instead.
|
|
13
|
+
|
|
14
|
+
## Contents
|
|
15
|
+
|
|
16
|
+
- [When this is the right tool (and when it is not)](#when-this-is-the-right-tool-and-when-it-is-not)
|
|
17
|
+
- [State record vs. lesson](#state-record-vs-lesson)
|
|
18
|
+
- [The cardinality rule](#the-cardinality-rule)
|
|
19
|
+
- [Conventions](#conventions)
|
|
20
|
+
- [The record shape](#the-record-shape)
|
|
21
|
+
- [Read step](#read-step)
|
|
22
|
+
- [Write step](#write-step)
|
|
23
|
+
- [TTL is a liveness guard](#ttl-is-a-liveness-guard)
|
|
24
|
+
- [Worked example — GitHub Actions flaky-test tracker](#worked-example--github-actions-flaky-test-tracker)
|
|
25
|
+
- [Guards (do not skip these)](#guards-do-not-skip-these)
|
|
26
|
+
- [The payoff — CI and agents share one store](#the-payoff--ci-and-agents-share-one-store)
|
|
27
|
+
- [Wiring checklist](#wiring-checklist)
|
|
28
|
+
|
|
29
|
+
---
|
|
30
|
+
|
|
31
|
+
## When this is the right tool (and when it is not)
|
|
32
|
+
|
|
33
|
+
Reach for a LoreKit state record when **both** hold:
|
|
34
|
+
|
|
35
|
+
- The job needs a **small, current** fact from previous runs — the latest value,
|
|
36
|
+
not a history.
|
|
37
|
+
- That fact is **also useful to an agent or a human**, not just to the next run of
|
|
38
|
+
the same job.
|
|
39
|
+
|
|
40
|
+
Use something else when:
|
|
41
|
+
|
|
42
|
+
| Need | Use instead |
|
|
43
|
+
| ---- | ----------- |
|
|
44
|
+
| Pure job-to-job caching, nobody else reads it | `actions/cache` — free, unlimited, built for this |
|
|
45
|
+
| A full run history / time series | Artifacts, a metrics backend, or your OTel pipeline |
|
|
46
|
+
| Large payloads (coverage reports, logs, traces) | Artifacts — the value cap is 64 KiB |
|
|
47
|
+
| A mutual-exclusion lock between concurrent jobs | A real lock (concurrency groups, an advisory lock) — writes here are last-write-wins with no compare-and-swap |
|
|
48
|
+
| Anything derived from a secret | Nothing. Do not store it. |
|
|
49
|
+
|
|
50
|
+
The second bullet of the "both" test is what earns LoreKit over `actions/cache`.
|
|
51
|
+
If only the next CI run will ever read it, `actions/cache` is the better answer and
|
|
52
|
+
this rule should say so out loud rather than sell the store.
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## State record vs. lesson
|
|
57
|
+
|
|
58
|
+
| | Lesson (`self-improvement-loops.md`) | CI state record (this file) |
|
|
59
|
+
| --- | --- | --- |
|
|
60
|
+
| Author | a model, at the end of a run | a script, deterministically |
|
|
61
|
+
| Value | prose + a `meta:` comment | JSON (an object, not a bare scalar) |
|
|
62
|
+
| How a reader uses it | **advisory** — a consideration that can be overridden | **authoritative** — parsed and branched on |
|
|
63
|
+
| Recurrence / promotion | yes (`seen_count`, human-gated hardening) | n/a — nothing is inferred, so nothing needs gating |
|
|
64
|
+
| Entrenchment risk | high — the reason those guards exist | none; the risks are cardinality and secrets instead |
|
|
65
|
+
| Key count over time | grows with distinct lessons | **fixed** — see below |
|
|
66
|
+
| Token | the agent's `lk_rw_*` | the job's `lk_wo_*` to write, `lk_ro_*` to read |
|
|
67
|
+
|
|
68
|
+
Because a state record is parsed rather than read, it must be **valid JSON on
|
|
69
|
+
every write** and **version-stamped**, so a reader written against v1 can detect a
|
|
70
|
+
v2 record instead of silently mis-parsing it.
|
|
71
|
+
|
|
72
|
+
---
|
|
73
|
+
|
|
74
|
+
## The cardinality rule
|
|
75
|
+
|
|
76
|
+
**One key per fact, overwritten in place. Never one key per run.**
|
|
77
|
+
|
|
78
|
+
This is the single constraint that keeps the idea sound, and it is easy to get
|
|
79
|
+
wrong — "record the state of the last run" reads like an append. It is not.
|
|
80
|
+
|
|
81
|
+
Same `scope` + `key` is an UPDATE, so a job that writes
|
|
82
|
+
`ci-state::flaky-tests` on every run holds **one** row forever, no matter how many
|
|
83
|
+
times it runs. A job that writes `ci-state::run-${{ github.run_id }}` adds a row
|
|
84
|
+
per run and will, in order: crowd the agent context window, blow the 5 000-memory
|
|
85
|
+
cap, and turn a memory store into a bad artifact bucket.
|
|
86
|
+
|
|
87
|
+
Three concrete limits make this a hard rule rather than a style preference:
|
|
88
|
+
|
|
89
|
+
- **64 KiB** per value (`MAX_VALUE_BYTES`) — a 400 above it.
|
|
90
|
+
- **5 000** active memories per user by default, enforced by a DB trigger.
|
|
91
|
+
- **120 requests/min** per user across every LoreKit surface.
|
|
92
|
+
|
|
93
|
+
And one soft limit that bites sooner: the agent SessionStart hook lists each scope
|
|
94
|
+
with a **read cap and no tag filter**, ordered by recency. Per-run CI writes are
|
|
95
|
+
the most recently updated rows in the repo scope, so they would displace the
|
|
96
|
+
lessons the hook exists to inject. Bounded cardinality is what keeps CI records
|
|
97
|
+
cheap enough to live in the same scope agents read.
|
|
98
|
+
|
|
99
|
+
---
|
|
100
|
+
|
|
101
|
+
## Conventions
|
|
102
|
+
|
|
103
|
+
Give the state its own bucket, in a namespace that can never be mistaken for a
|
|
104
|
+
lesson bucket:
|
|
105
|
+
|
|
106
|
+
- **Tag:** `ci::<job>-state` — e.g. `ci::test-state`, `ci::deploy-state`.
|
|
107
|
+
Deliberately **not** `loop::…`; that prefix is the lessons grammar.
|
|
108
|
+
- **Key:** `ci-state::<slug>` — e.g. `ci-state::flaky-tests`. One slug per fact.
|
|
109
|
+
- **Taxonomy:** pass `--kind bus --host ci` explicitly. `kind`/`host` are only
|
|
110
|
+
inferred from `loop::` tags, so a `ci::` tag leaves them NULL unless you say so.
|
|
111
|
+
Setting them buys `lorekit list --kind bus --host ci` — one command that shows
|
|
112
|
+
every state record and nothing else.
|
|
113
|
+
|
|
114
|
+
Scope, by what the fact is about:
|
|
115
|
+
|
|
116
|
+
- **`repo::{owner}/{repo}`** — the default. Trunk state: flaky tests, the last
|
|
117
|
+
deployed SHA, a benchmark baseline.
|
|
118
|
+
- **`branch::{owner}/{repo}::{branch}`** — per-PR state that should disappear with
|
|
119
|
+
the branch (what this PR's last run already commented on). Pair it with
|
|
120
|
+
`--ttl-days` so it self-cleans.
|
|
121
|
+
- **`global`** — almost never. A CI fact is repo-bound by construction.
|
|
122
|
+
|
|
123
|
+
---
|
|
124
|
+
|
|
125
|
+
## The record shape
|
|
126
|
+
|
|
127
|
+
A JSON object with a version stamp and a provenance block, so a reader can tell
|
|
128
|
+
which run produced it and whether it understands the format:
|
|
129
|
+
|
|
130
|
+
```json
|
|
131
|
+
{
|
|
132
|
+
"v": 1,
|
|
133
|
+
"updated_by_run": "https://github.com/owner/repo/actions/runs/123456789",
|
|
134
|
+
"commit": "0f4a1c9…",
|
|
135
|
+
"data": {
|
|
136
|
+
"flaky": ["src/queue.test.ts::retries on 429", "src/auth.test.ts::refresh"],
|
|
137
|
+
"consecutive_green": 3
|
|
138
|
+
}
|
|
139
|
+
}
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
Rules that make it safe to parse:
|
|
143
|
+
|
|
144
|
+
- `v` is required and bumped on any breaking shape change. A reader that sees an
|
|
145
|
+
unknown `v` **falls back to its first-run path and logs it** — it never guesses.
|
|
146
|
+
- Everything mutable lives under `data`, so the envelope stays stable.
|
|
147
|
+
- No secrets, no tokens, no full environment dumps, no raw log bodies (see
|
|
148
|
+
[Guards](#guards-do-not-skip-these)).
|
|
149
|
+
|
|
150
|
+
---
|
|
151
|
+
|
|
152
|
+
## Read step
|
|
153
|
+
|
|
154
|
+
Address the record with **`--scope` and `--key` flags, not the single-token
|
|
155
|
+
`<scope::key>` form** — the key itself contains `::`, and flags are the only
|
|
156
|
+
unambiguous way to express that.
|
|
157
|
+
|
|
158
|
+
```bash
|
|
159
|
+
set -euo pipefail
|
|
160
|
+
|
|
161
|
+
STATE_JSON='{}'
|
|
162
|
+
if lorekit show --scope "repo::${REPO}" --key 'ci-state::flaky-tests' \
|
|
163
|
+
--remote --json > state-raw.json 2>&1; then
|
|
164
|
+
cat state-raw.json # the log-visibility rule: always echo
|
|
165
|
+
STATE_JSON=$(jq -r '.remote.record.value // "{}"' state-raw.json)
|
|
166
|
+
else
|
|
167
|
+
cat state-raw.json
|
|
168
|
+
echo "No prior state (first run, or LoreKit unreachable) — continuing with defaults."
|
|
169
|
+
fi
|
|
170
|
+
|
|
171
|
+
VERSION=$(jq -r '.v // 0' <<<"$STATE_JSON")
|
|
172
|
+
if [ "$VERSION" != "1" ]; then
|
|
173
|
+
echo "State schema v${VERSION} is not v1 — ignoring it and rebuilding from scratch."
|
|
174
|
+
STATE_JSON='{}'
|
|
175
|
+
fi
|
|
176
|
+
```
|
|
177
|
+
|
|
178
|
+
Two things this deliberately gets right:
|
|
179
|
+
|
|
180
|
+
- **`lorekit show` exits 1 on a miss.** That is not an error condition here — the
|
|
181
|
+
first run of any new state record misses. Branch on it; do not `|| true` it away,
|
|
182
|
+
which would swallow a genuine auth or network failure too.
|
|
183
|
+
- **A LoreKit outage degrades to the first-run path.** The job continues; it does
|
|
184
|
+
not fail. See [Guards](#guards-do-not-skip-these).
|
|
185
|
+
|
|
186
|
+
---
|
|
187
|
+
|
|
188
|
+
## Write step
|
|
189
|
+
|
|
190
|
+
```bash
|
|
191
|
+
set -euo pipefail
|
|
192
|
+
|
|
193
|
+
jq -nc \
|
|
194
|
+
--arg run "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" \
|
|
195
|
+
--arg sha "${GITHUB_SHA}" \
|
|
196
|
+
--argjson data "$NEW_DATA" \
|
|
197
|
+
'{v: 1, updated_by_run: $run, commit: $sha, data: $data}' \
|
|
198
|
+
| lorekit write \
|
|
199
|
+
--scope "repo::${REPO}" \
|
|
200
|
+
--key 'ci-state::flaky-tests' \
|
|
201
|
+
--tags 'ci::test-state' \
|
|
202
|
+
--kind bus --host ci \
|
|
203
|
+
--ttl-days 7 \
|
|
204
|
+
--remote --json \
|
|
205
|
+
| tee write-result.json
|
|
206
|
+
```
|
|
207
|
+
|
|
208
|
+
Notes on each flag that is not obvious:
|
|
209
|
+
|
|
210
|
+
- **`--ttl-days` is not optional, and it should be short.** See
|
|
211
|
+
[TTL is a liveness guard](#ttl-is-a-liveness-guard) — the countdown refreshes on
|
|
212
|
+
every write, so a short TTL expires the record only when the *job* stops running.
|
|
213
|
+
Never omit it: a write passing neither `--ttl-days` nor `--clear-ttl` inherits
|
|
214
|
+
whatever `ttl.default` / `scope.defaults.<prefix>.ttl_days` the repo config
|
|
215
|
+
happens to set for lessons, which is a number nobody chose for this record.
|
|
216
|
+
- **No value argument** — `lorekit write` reads stdin when none is given, which is
|
|
217
|
+
what makes the `jq | lorekit write` pipe work.
|
|
218
|
+
- **`--remote`** — be explicit. A CI checkout has no offline store, and picking the
|
|
219
|
+
target by inference is not something to leave to chance in a pipeline.
|
|
220
|
+
- **`--kind bus --host ci`** — see [Conventions](#conventions).
|
|
221
|
+
|
|
222
|
+
The token comes from `LOREKIT_TOKEN` in the environment. Use a **write-only
|
|
223
|
+
`lk_wo_*` token for the writing job**: it cannot read anything back, so a leaked CI
|
|
224
|
+
token cannot exfiltrate the team's lore. Where the same job must also read, it needs
|
|
225
|
+
`lk_rw_*` — split the steps and use two tokens if the read is small enough to
|
|
226
|
+
justify it.
|
|
227
|
+
|
|
228
|
+
Over REST instead of the CLI, the equivalents are `POST /memories` (write) and
|
|
229
|
+
`GET /memories?scope=…&key=…` (read), same auth header. The CLI is preferred in CI
|
|
230
|
+
because it derives `origin` (repo / branch / commit / PR) from the GitHub Actions
|
|
231
|
+
environment automatically, which is what makes each record traceable back to the
|
|
232
|
+
run that wrote it in the dashboard.
|
|
233
|
+
|
|
234
|
+
---
|
|
235
|
+
|
|
236
|
+
## TTL is a liveness guard
|
|
237
|
+
|
|
238
|
+
**Default to a short TTL — roughly a week — not a permanent record.** This is the
|
|
239
|
+
opposite of the instinct that "state must persist", and it follows from one detail
|
|
240
|
+
of the write path.
|
|
241
|
+
|
|
242
|
+
`memory_write` computes `expires_at = now() + ttl_days` on **every** write, insert
|
|
243
|
+
or update (`supabase/migrations/00030_memory_ttl.sql`: `expires_at = case when
|
|
244
|
+
p_ttl_days is not null then v_expires_at else memories.expires_at end`). The job
|
|
245
|
+
rewrites its record on every run and passes `--ttl-days` every time, so the
|
|
246
|
+
countdown restarts every run.
|
|
247
|
+
|
|
248
|
+
That makes the TTL measure **how long the job has been silent**, not how old the
|
|
249
|
+
record is. A record expires when — and only when — the job stopped running for that
|
|
250
|
+
long, which is exactly when its contents stopped being true. A daily job with a
|
|
251
|
+
7-day TTL keeps its state indefinitely while it runs daily, and drops it a week
|
|
252
|
+
after someone deletes the workflow.
|
|
253
|
+
|
|
254
|
+
| Job cadence | TTL | Why |
|
|
255
|
+
| ----------- | --- | --- |
|
|
256
|
+
| Every push / per-PR | **7 days** | Survives a feature freeze or a quiet holiday week; an abandoned job self-cleans |
|
|
257
|
+
| Nightly | **14 days** | Tolerates a fortnight of red or paused schedules |
|
|
258
|
+
| Weekly (release, audit) | **30 days** | ~4 missed runs of slack |
|
|
259
|
+
| Branch-scoped, any cadence | **7 days or less** | The branch outlives the state; let it decay with the PR |
|
|
260
|
+
|
|
261
|
+
Pick the number so a *normal* quiet spell does not expire the record, and an
|
|
262
|
+
abandoned job does. Do not reach for 365 to be safe — that is just "permanent" with
|
|
263
|
+
extra steps, and it re-creates the failure below.
|
|
264
|
+
|
|
265
|
+
Two things this buys beyond freshness:
|
|
266
|
+
|
|
267
|
+
- **A stale record is worse than a missing one.** Falling back to the first-run path
|
|
268
|
+
is a defined, tested code path. Acting on a flaky-test set from four months ago is
|
|
269
|
+
not — it is silently wrong, and nothing surfaces that.
|
|
270
|
+
- **It bounds the blast radius of a cardinality mistake.** [The cardinality
|
|
271
|
+
rule](#the-cardinality-rule) is a discipline, and disciplines get violated. If
|
|
272
|
+
someone keys on `github.run_id` anyway, a 7-day TTL turns unbounded growth into a
|
|
273
|
+
bounded steady state that drains itself — the store stops filling instead of
|
|
274
|
+
climbing to the 5 000-memory cap.
|
|
275
|
+
|
|
276
|
+
**`--clear-ttl` is the rare exception, not the default.** Reserve it for a record
|
|
277
|
+
whose *absence* is more dangerous than its staleness — a migration watermark, say.
|
|
278
|
+
If you find yourself there, ask first whether the fact belongs in a best-effort
|
|
279
|
+
memory store at all rather than in the datastore that owns it.
|
|
280
|
+
|
|
281
|
+
---
|
|
282
|
+
|
|
283
|
+
## Worked example — GitHub Actions flaky-test tracker
|
|
284
|
+
|
|
285
|
+
Applies the usual workflow-authoring rules (the `github-actions-author` skill in
|
|
286
|
+
[mthines/agent-skills](https://github.com/mthines/agent-skills) is the reference):
|
|
287
|
+
named steps, least-privilege `permissions`, SHA-pinned third-party actions,
|
|
288
|
+
`concurrency` without cancellation (a cancelled run must not leave half-written
|
|
289
|
+
state), `set -euo pipefail`, and every command's output reaching the run log.
|
|
290
|
+
|
|
291
|
+
```yaml
|
|
292
|
+
name: Tests
|
|
293
|
+
|
|
294
|
+
on:
|
|
295
|
+
push:
|
|
296
|
+
branches: [main]
|
|
297
|
+
paths: ['src/**', 'package-lock.json', '.github/workflows/tests.yml']
|
|
298
|
+
|
|
299
|
+
permissions:
|
|
300
|
+
contents: read
|
|
301
|
+
|
|
302
|
+
concurrency:
|
|
303
|
+
group: tests-${{ github.ref }}
|
|
304
|
+
cancel-in-progress: false # never abandon a run mid-state-write
|
|
305
|
+
|
|
306
|
+
jobs:
|
|
307
|
+
test:
|
|
308
|
+
runs-on: ubuntu-latest
|
|
309
|
+
env:
|
|
310
|
+
LOREKIT_TOKEN: ${{ secrets.LOREKIT_TOKEN_RW }}
|
|
311
|
+
REPO: ${{ github.repository }}
|
|
312
|
+
steps:
|
|
313
|
+
- name: Check out the repository
|
|
314
|
+
uses: actions/checkout@11bd71901bbe5b1630ceea73d27597364c9af683 # v4.2.2
|
|
315
|
+
|
|
316
|
+
- name: Set up Node.js
|
|
317
|
+
uses: actions/setup-node@39370e3970a6d050c480ffad4ff0ed4d3fdee5af # v4.1.0
|
|
318
|
+
with:
|
|
319
|
+
node-version: 22
|
|
320
|
+
cache: npm
|
|
321
|
+
|
|
322
|
+
- name: Install dependencies
|
|
323
|
+
run: |
|
|
324
|
+
set -euo pipefail
|
|
325
|
+
npm ci 2>&1 | tee install.log
|
|
326
|
+
|
|
327
|
+
- name: Read the known-flaky set from LoreKit
|
|
328
|
+
run: |
|
|
329
|
+
set -euo pipefail
|
|
330
|
+
KNOWN='[]'
|
|
331
|
+
if npx --yes @lorekit/cli@latest show \
|
|
332
|
+
--scope "repo::${REPO}" --key 'ci-state::flaky-tests' \
|
|
333
|
+
--remote --json > state.json 2>&1; then
|
|
334
|
+
cat state.json
|
|
335
|
+
KNOWN=$(jq -c '(.remote.record.value // "{}") | fromjson
|
|
336
|
+
| if .v == 1 then .data.flaky else [] end' state.json)
|
|
337
|
+
else
|
|
338
|
+
cat state.json
|
|
339
|
+
echo "No prior flaky state — treating every failure as new."
|
|
340
|
+
fi
|
|
341
|
+
echo "KNOWN_FLAKY=${KNOWN}" >> "$GITHUB_ENV"
|
|
342
|
+
echo "Known flaky on entry: ${KNOWN}"
|
|
343
|
+
|
|
344
|
+
- name: Run the test suite
|
|
345
|
+
id: tests
|
|
346
|
+
run: |
|
|
347
|
+
set -euo pipefail
|
|
348
|
+
npm test -- --reporter=default --reporter=json --outputFile=results.json 2>&1 \
|
|
349
|
+
| tee test-output.log
|
|
350
|
+
|
|
351
|
+
- name: Record the flaky set back to LoreKit
|
|
352
|
+
if: always()
|
|
353
|
+
run: |
|
|
354
|
+
set -euo pipefail
|
|
355
|
+
FLAKY=$(jq -c '[.testResults[]? | select(.status == "failed") | .name]' results.json)
|
|
356
|
+
jq -nc \
|
|
357
|
+
--arg run "${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}/actions/runs/${GITHUB_RUN_ID}" \
|
|
358
|
+
--arg sha "${GITHUB_SHA}" \
|
|
359
|
+
--argjson flaky "$FLAKY" \
|
|
360
|
+
'{v: 1, updated_by_run: $run, commit: $sha, data: {flaky: $flaky}}' \
|
|
361
|
+
| npx --yes @lorekit/cli@latest write \
|
|
362
|
+
--scope "repo::${REPO}" --key 'ci-state::flaky-tests' \
|
|
363
|
+
--tags 'ci::test-state' --kind bus --host ci \
|
|
364
|
+
--ttl-days 7 --remote --json \
|
|
365
|
+
| tee lorekit-write.log \
|
|
366
|
+
|| echo "LoreKit write failed (exit $?) — not failing the build; see the output above."
|
|
367
|
+
```
|
|
368
|
+
|
|
369
|
+
The last step is `if: always()` and its failure is swallowed **with the output
|
|
370
|
+
echoed and the exit code named** — a memory store being down is never a reason to
|
|
371
|
+
turn a green test run red, but a silent `|| true` would hide a misconfigured token
|
|
372
|
+
for months.
|
|
373
|
+
|
|
374
|
+
---
|
|
375
|
+
|
|
376
|
+
## Guards (do not skip these)
|
|
377
|
+
|
|
378
|
+
The lessons loop's entrenchment guards do not apply — nothing here is inferred. These
|
|
379
|
+
do:
|
|
380
|
+
|
|
381
|
+
1. **Bounded cardinality.** One key per fact, overwritten in place. If the number
|
|
382
|
+
of keys a job writes grows with the number of runs, the design is wrong. See
|
|
383
|
+
[The cardinality rule](#the-cardinality-rule).
|
|
384
|
+
2. **No secrets, ever — and the bar is higher here than for lessons.** A lesson is
|
|
385
|
+
drafted by a model that can be told to redact; a state record is assembled
|
|
386
|
+
mechanically from a CI environment that is *full* of tokens. Never
|
|
387
|
+
`env | jq -R`, never a raw log body, never an error string that might carry a
|
|
388
|
+
connection URL. Build the payload from an explicit allow-list of fields.
|
|
389
|
+
3. **Expiry is explicit, and short by default.** Pass `--ttl-days` sized to the
|
|
390
|
+
job's cadence (~7 days for anything running at least daily). It refreshes on
|
|
391
|
+
every write, so it expires only when the job goes silent — see
|
|
392
|
+
[TTL is a liveness guard](#ttl-is-a-liveness-guard). Never inherit the config
|
|
393
|
+
default by accident, and reserve `--clear-ttl` for the rare record whose absence
|
|
394
|
+
is worse than its staleness.
|
|
395
|
+
4. **The store is never on the critical path.** A read miss falls back to the
|
|
396
|
+
first-run path; a write failure is logged and swallowed. A LoreKit outage must
|
|
397
|
+
not fail a build — but it must be *visible* in the log, never `|| true`-d away.
|
|
398
|
+
5. **Last write wins; there is no compare-and-swap.** Two concurrent jobs writing
|
|
399
|
+
the same key will clobber each other. Either write from one job, or serialise
|
|
400
|
+
with a `concurrency` group, or shard the key.
|
|
401
|
+
6. **Version every record.** An unrecognised `v` means fall back to the first-run
|
|
402
|
+
path and log it — never parse a shape you do not recognise.
|
|
403
|
+
7. **Remember agents read this scope.** State records land in `repo::` alongside
|
|
404
|
+
lessons and will surface in an agent's SessionStart injection. That is the point
|
|
405
|
+
(see below) — but it means the value should read sensibly to a human skimming
|
|
406
|
+
it, and stay small.
|
|
407
|
+
|
|
408
|
+
---
|
|
409
|
+
|
|
410
|
+
## The payoff — CI and agents share one store
|
|
411
|
+
|
|
412
|
+
Job-to-job persistence alone does not justify LoreKit over `actions/cache`. What
|
|
413
|
+
does is that **both sides of the loop read the same store**:
|
|
414
|
+
|
|
415
|
+
- CI writes `ci-state::flaky-tests` deterministically on every run of `main`.
|
|
416
|
+
- An agent asked to "fix the flaky tests" reads it at SessionStart — the same repo
|
|
417
|
+
scope, no extra wiring — and starts from the real list instead of re-deriving it.
|
|
418
|
+
- The agent's own findings go back as a **lesson** in the `loop::` bucket, in the
|
|
419
|
+
same repo scope, where the next human and the next agent both see it.
|
|
420
|
+
- The dashboard shows both, with each state record linked back to the exact
|
|
421
|
+
workflow run that wrote it via the derived `origin`.
|
|
422
|
+
|
|
423
|
+
So the two record kinds are complementary, not competing: the state record carries
|
|
424
|
+
**what is true right now**, the lesson carries **what we learned about it**. Keep
|
|
425
|
+
them in separate buckets (`ci::` vs `loop::`) so each can be read, filtered, and
|
|
426
|
+
groomed on its own terms.
|
|
427
|
+
|
|
428
|
+
---
|
|
429
|
+
|
|
430
|
+
## Wiring checklist
|
|
431
|
+
|
|
432
|
+
To add CI state to a job called `<job>`:
|
|
433
|
+
|
|
434
|
+
- [ ] Confirm the fact is small, current-only, and useful to more than the next CI
|
|
435
|
+
run — otherwise use `actions/cache`.
|
|
436
|
+
- [ ] Pick the bucket: tag `ci::<job>-state`, key `ci-state::<slug>`, one slug per
|
|
437
|
+
fact, `--kind bus --host ci`.
|
|
438
|
+
- [ ] Pick the scope: `repo::` for trunk state, `branch::` for per-PR state.
|
|
439
|
+
- [ ] Pick the TTL from the job's cadence (7 / 14 / 30 days — see the table). Short
|
|
440
|
+
by default; `--clear-ttl` only with a reason.
|
|
441
|
+
- [ ] Define the JSON envelope with a `v` stamp, and the reader's unknown-version
|
|
442
|
+
fallback.
|
|
443
|
+
- [ ] Add the **read step** early in the job (`--scope`/`--key` flags, exit-1-is-a-miss
|
|
444
|
+
branch, output echoed).
|
|
445
|
+
- [ ] Add the **write step** at the end (`if: always()` if the state should survive a
|
|
446
|
+
failing run), `--ttl-days` passed on EVERY write, failure swallowed but logged.
|
|
447
|
+
- [ ] Provision tokens: `lk_wo_*` for a write-only job, `lk_ro_*` for a read-only
|
|
448
|
+
one, `lk_rw_*` only where one job genuinely needs both.
|
|
449
|
+
- [ ] Verify the payload is built from an explicit allow-list of fields, with no
|
|
450
|
+
environment dump and no raw error bodies.
|
|
451
|
+
- [ ] Confirm the key count does not grow with run count — run the job twice and
|
|
452
|
+
check `lorekit list --kind bus --host ci` still shows one row per fact.
|
|
@@ -46,6 +46,10 @@ Do **not** add a loop to:
|
|
|
46
46
|
must not be biased by prior runs; that is the whole point of a fresh pass.
|
|
47
47
|
- **Steps that handle secrets or credentials** — routing them through lesson
|
|
48
48
|
extraction is a leak risk.
|
|
49
|
+
- **Deterministic jobs** (a CI workflow, a cron script, a release pipeline) — they
|
|
50
|
+
do not draft prose lessons or need a promotion gate; what they want is a JSON
|
|
51
|
+
**state record** of their last run. That is a different shape with different
|
|
52
|
+
guards: see [ci-state-records.md](./ci-state-records.md).
|
|
49
53
|
|
|
50
54
|
When unsure, default to **no**. A loop can be added later; unwinding an
|
|
51
55
|
entrenched-bias loop is harder.
|