pi-sleep-agent 0.8.16 → 0.8.18
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +15 -2
- package/docs/EVIDENCE.md +64 -0
- package/package.json +1 -1
- package/sleepkit/cli.py +54 -7
- package/sleepkit/consolidate.py +53 -2
- package/sleepkit/store.py +62 -7
package/README.md
CHANGED
|
@@ -27,6 +27,11 @@ Sleep adds a reviewable memory cycle to nine selectable coding-agent targets. It
|
|
|
27
27
|
|
|
28
28
|
- **Automatic capture.** Hooks and the Pi extension collect useful observations while you work.
|
|
29
29
|
- **Budgeted consolidation.** Sleep runs only after the record and time gates pass.
|
|
30
|
+
- **A named consolidation window.** One pass is shown the newest 120 T0 records. A buffer
|
|
31
|
+
past that is never silent: `distill --dry-run` and the pass itself report how many records
|
|
32
|
+
fell outside the window, `status` says it before the next pass runs, and `doctor` reports
|
|
33
|
+
it as its own finding. Records outside the window are shown to no pass and cleared by no
|
|
34
|
+
prune, which is why the number is printed instead of being inferred from the proposal count.
|
|
30
35
|
- **Review before permanence.** New memories stay provisional until you promote or refuse them.
|
|
31
36
|
- **Nightly dream cycle.** Mine repeated preferences, consolidate recent observations, and measure memory effects while you are away.
|
|
32
37
|
- **Measured replay.** Run the same task with memory off and on before deciding whether a memory helps. Each completed pass leaves a one-line quality summary (`helps / hurts / ties`) in `status` and in the nightly report, so the question stays answerable between passes; a pass whose two arms ran the same prompt says it measured nothing instead of reporting zeros.
|
|
@@ -38,7 +43,13 @@ Sleep adds a reviewable memory cycle to nine selectable coding-agent targets. It
|
|
|
38
43
|
the thing it describes actually happened, and it is written **only when the record
|
|
39
44
|
states a date** (a date equal to the capture day is not a second clock). An empty
|
|
40
45
|
`event_at` means the record does not say, never a guess, so `show` can answer
|
|
41
|
-
"when was this true" without inventing an event time.
|
|
46
|
+
"when was this true" without inventing an event time. Both clocks are read, not just
|
|
47
|
+
stored: when `doctor` flags a contradicting pair it names the older side as the claim
|
|
48
|
+
on the pair's `event_at` (falling back to `ts` when neither record states a date), so
|
|
49
|
+
a fact you file today about a state that was already superseded is not handed back as
|
|
50
|
+
the newer claim; injection appends `(described a state Nd old)` when the two clocks
|
|
51
|
+
differ, and the adjudicator model is shown both, named as older or newer than the
|
|
52
|
+
capture.
|
|
42
53
|
- **Tagged records.** Consolidation writes 1-4 lowercase topic tags per record (`deploy|aws`),
|
|
43
54
|
normalized and capped in `store`, and `inject --topic` selects on them. A tag is a
|
|
44
55
|
hint a model proposed, so a wrong tag costs a missing filter hit, never a wrong fact.
|
|
@@ -330,7 +341,9 @@ Run the two dream components separately when you want more control:
|
|
|
330
341
|
| `./sleep.py status -v` | ... and the last measured memory quality (`3 helps / 1 hurts / 4 ties from 8 samples`, or "no replay measurement has run yet") |
|
|
331
342
|
| `./sleep.py mine --dry-run` | Find preferences repeated across sessions |
|
|
332
343
|
| `./sleep.py nightly --dry-run` | Preview the unattended nightly workflow |
|
|
333
|
-
| `./sleep.py doctor` | Find duplicate or overlapping memory records, flag contradicting candidates (same subject, one side carries a correction marker) and print the `supersede <older> <newer>` command to run, flag provisional candidates not rediscovered for 4+ weeks, flag records no session was ever served for 8+ weeks, flag supersede pointers that name a record no longer in the store,
|
|
344
|
+
| `./sleep.py doctor` | Find duplicate or overlapping memory records, flag contradicting candidates (same subject, one side carries a correction marker) and print the `supersede <older> <newer>` command to run, flag provisional candidates not rediscovered for 4+ weeks, flag records no session was ever served for 8+ weeks, flag supersede pointers that name a record no longer in the store, report a T0 buffer larger
|
|
345
|
+
than the 120-record consolidation window (with the count left outside it), and report records
|
|
346
|
+
parked in quarantine awaiting a human (report only: it never supersedes, drops, merges or un-parks anything itself) |
|
|
334
347
|
| `./sleep.py doctor --stale <value>` | Show which approved records still hold a stale value (e.g. a changed URL) |
|
|
335
348
|
| `./sleep.py doctor --contradictions` | Ask a model to adjudicate shortlisted pairs of provisional records for conflicts (`--budget`, default $0.25); report-only, and each finding carries the `supersede <older> <newer>` command to run |
|
|
336
349
|
| `./sleep.py doctor --contradictions --tier t2` | Adjudicate pairs of **approved** permanent memory instead of candidates (`--tier both` does both); a T2 finding is labelled `contradicting-memory` and carries the `demote <older>` command, because approved memory is un-approved, never superseded |
|
package/docs/EVIDENCE.md
CHANGED
|
@@ -467,3 +467,67 @@ used `===PROPOSAL===` only as a closing marker. The parser discarded everything
|
|
|
467
467
|
marker. `sleepkit.consolidate.split_blocks()` now preserves a leading contract-shaped proposal while
|
|
468
468
|
still discarding ordinary preamble, pinned by
|
|
469
469
|
`test_closing_proposal_marker_preserves_leading_proposal`.
|
|
470
|
+
|
|
471
|
+
|
|
472
|
+
## 2026-11-03: world-time ordering for contradicting records
|
|
473
|
+
|
|
474
|
+
`doctor` named the older side of a contradicting pair by capture time (`ts`), which is the
|
|
475
|
+
store's clock and not the claim's. An agent that files an old fact today - a document quote
|
|
476
|
+
saying the deploy host is X, recorded yesterday, while a person corrected it in the store a week
|
|
477
|
+
ago - therefore looked like the *newer* claim, and the suggested command
|
|
478
|
+
(`supersede <older> <newer>`) pointed the wrong way.
|
|
479
|
+
|
|
480
|
+
The pair now carries both clocks as `(ts, event_at)`, in the T1 shortlist and in
|
|
481
|
+
`permanent_records`, and one axis decides each comparison: `event_at` when both records state a
|
|
482
|
+
date, the `ts` pair otherwise, the id when neither does. Each comparison runs on a single axis
|
|
483
|
+
because the first version preferred each record's own clock (`ta or ea`), which is just `ts`
|
|
484
|
+
again for any record carrying both, and it reproduced the inverted advice the change exists to
|
|
485
|
+
fix. `inject` appends `(described a state Nd old)` only when the two clocks differ, and the
|
|
486
|
+
adjudicator prompt now shows both clocks with their relation
|
|
487
|
+
(`event_at=2026-10-01 (older state, recorded 2026-11-02)`).
|
|
488
|
+
|
|
489
|
+
Pinned by `test_a_late_note_about_an_old_state_is_named_the_claim_not_the_correction`,
|
|
490
|
+
`test_without_a_stated_date_the_capture_order_still_decides`,
|
|
491
|
+
`test_the_adjudicator_is_shown_both_clocks_and_their_relation` (all in `tests/test_cli.py`,
|
|
492
|
+
crossing a process boundary with a prompt-recording fake model),
|
|
493
|
+
`test_permanent_records_carry_both_clocks_not_a_bare_ts` and
|
|
494
|
+
`test_ordering_prefers_the_stated_date_and_falls_back_to_capture_time` (`tests/test_core.py`).
|
|
495
|
+
|
|
496
|
+
Not changed, and named here so the leftover is not mistaken for coverage: `candidate_pairs()`
|
|
497
|
+
still orders by `ts` and still makes one model call per pair, and `cmd_doctor` does not call it.
|
|
498
|
+
It is a duplicate of the ordering rule with a per-pair cost, left alone deliberately.
|
|
499
|
+
|
|
500
|
+
|
|
501
|
+
## 2026-11-03: the consolidation window is named, and the record left outside it is counted
|
|
502
|
+
|
|
503
|
+
`_fmt_buffer` showed a consolidation pass only the newest 120 T0 records. Everything older was
|
|
504
|
+
never read by any pass, and because pruning only clears what a pass consumed, never cleared either.
|
|
505
|
+
Nothing said so: the night reported `proposals: 2` whether the buffer held 12 records or 300, and
|
|
506
|
+
that number reads as a complete pass.
|
|
507
|
+
|
|
508
|
+
Measured before changing anything, and the honest reading is that the cap had never bitten here:
|
|
509
|
+
`~/.sleep/archive/buffer` holds 31 consumed buffers totalling 259 records, the largest a single
|
|
510
|
+
night ever held was 73, and the live buffer held 1. So this was a latent silent loss, not an
|
|
511
|
+
observed one, and the change is about the report rather than about the limit.
|
|
512
|
+
|
|
513
|
+
What the window now costs to see, all three surfaces checked against a real 127-record buffer in a
|
|
514
|
+
throwaway `SLEEP_HOME`:
|
|
515
|
+
|
|
516
|
+
| surface | without the window | with records outside it |
|
|
517
|
+
|---|---|---|
|
|
518
|
+
| `distill --dry-run` / the pass result | key absent | `outside_window: 7`, `warn: "7 of 127 buffer records fell outside the 120-record window and were not read or pruned"` |
|
|
519
|
+
| `sleep.py status` | T0 line unchanged | `T0 buffer 127 records ... [next pass reads the newest 120; 7 outside the window]` |
|
|
520
|
+
| `sleep.py doctor` | no finding | `{"issue": "buffer-exceeds-consolidation-window", "buffer_records": 127, "window": 120, "outside_window": 7}` |
|
|
521
|
+
|
|
522
|
+
`run_sleep` now persists `buffer_records` and `outside_window` in `state.json` next to `proposals`,
|
|
523
|
+
so a night that left records behind is readable after the fact. The window lives in one place,
|
|
524
|
+
`sleepkit.consolidate.BUFFER_WINDOW`, and the sentence is built by one helper (`_window_warning`,
|
|
525
|
+
wrapped by `_warn_field` so the key is absent rather than empty when there is nothing to report) so
|
|
526
|
+
the preview and the real pass cannot drift. Pinned by `tests/test_cli.py::BufferWindow` (5 tests,
|
|
527
|
+
crossing a process boundary): under the window nothing appears in any of the three surfaces, at
|
|
528
|
+
`WINDOW + 7` the count and the sentence say both numbers, and a pass the record/time gate *refuses*
|
|
529
|
+
still reports the buffer size it did not read.
|
|
530
|
+
|
|
531
|
+
Deliberately NOT changed, since the two are separable and only the reporting was asked for: the pass
|
|
532
|
+
still reads only the newest window and the rest still waits for the next night. Sending a large
|
|
533
|
+
buffer in several passes is a separate change with its own cost (one model call per slice).
|
package/package.json
CHANGED
package/sleepkit/cli.py
CHANGED
|
@@ -70,7 +70,14 @@ def cmd_status(a) -> int:
|
|
|
70
70
|
_p(replay_mod.quality_line())
|
|
71
71
|
except Exception:
|
|
72
72
|
pass
|
|
73
|
-
|
|
73
|
+
line = "T0 buffer %d records / %d files / %d bytes (captured while you worked)" % (bc["records"], bc["files"], bc["bytes"])
|
|
74
|
+
# A window smaller than the buffer is the one case where the next pass cannot read
|
|
75
|
+
# everything it is standing on. Say it up front instead of leaving it to be inferred
|
|
76
|
+
# from the proposals count of a night that already happened.
|
|
77
|
+
if bc["records"] > consolidate.BUFFER_WINDOW:
|
|
78
|
+
line += " [next pass reads the newest %d; %d outside the window]" % (
|
|
79
|
+
consolidate.BUFFER_WINDOW, bc["records"] - consolidate.BUFFER_WINDOW)
|
|
80
|
+
_p(line)
|
|
74
81
|
_p("T1 candidates %d provisional, %d refused, %d total (distilled, shown as unverified context)" % (len(prov), len(ref), len(cands)))
|
|
75
82
|
_p(" quarantine %d parked (needs human eyes) (held out until you review)" % len(q))
|
|
76
83
|
retracted = []
|
|
@@ -208,10 +215,16 @@ def cmd_prune(a) -> int:
|
|
|
208
215
|
|
|
209
216
|
|
|
210
217
|
def _doctor_shortlist() -> list:
|
|
211
|
-
"""(id, body, ts) for
|
|
218
|
+
"""(id, body, (ts, event_at)) for provisional records, the input to candidate_pairs.
|
|
219
|
+
|
|
220
|
+
Both clocks travel: `ts` is when the store first held the claim, `event_at` is when
|
|
221
|
+
the record says the described thing was true. Deciding which of two records is the
|
|
222
|
+
older claim is a world-time question, and a session that files an old fact today
|
|
223
|
+
makes ts alone pick the wrong side."""
|
|
212
224
|
out = []
|
|
213
225
|
for c in store.list_candidates(status="provisional", with_body=True):
|
|
214
|
-
out.append((c["id"], c.get("_body", "") or "",
|
|
226
|
+
out.append((c["id"], c.get("_body", "") or "",
|
|
227
|
+
(str(c.get("ts", "")), str(c.get("event_at", "")))))
|
|
215
228
|
return out
|
|
216
229
|
|
|
217
230
|
|
|
@@ -248,8 +261,18 @@ def _llm_contradictions(pairs: list, budget: float, model: str | None,
|
|
|
248
261
|
lines.append("Pair %d:" % i)
|
|
249
262
|
for cid in (older, newer):
|
|
250
263
|
meta, body = seen.get(cid, ({}, ""))
|
|
251
|
-
|
|
252
|
-
|
|
264
|
+
# Both clocks and the relation between them: a record learned long after the
|
|
265
|
+
# state it describes is the late note here, and a model shown only `ts` would
|
|
266
|
+
# read it as the newer claim and propose superseding the true memory.
|
|
267
|
+
ts = str(meta.get("ts", "?"))
|
|
268
|
+
event_at = str(meta.get("event_at", "") or "")
|
|
269
|
+
if event_at and event_at[:10] != ts[:10]:
|
|
270
|
+
when = "ts=%s event_at=%s (%s state, recorded %s)" % (
|
|
271
|
+
ts, event_at, "older" if event_at < ts else "newer", ts[:10])
|
|
272
|
+
else:
|
|
273
|
+
when = "ts=%s event_at=%s" % (ts, event_at or "unknown")
|
|
274
|
+
lines.append(" id=%s %s body=%s" % (cid, when,
|
|
275
|
+
(body or "").replace("\n", " ").strip()))
|
|
253
276
|
if tier in ("t2", "both"):
|
|
254
277
|
user = ("Memory records, grouped into candidate pairs. Some are unreviewed "
|
|
255
278
|
"provisional candidates; some are permanent memories a human already "
|
|
@@ -321,7 +344,7 @@ def _llm_contradictions(pairs: list, budget: float, model: str | None,
|
|
|
321
344
|
|
|
322
345
|
|
|
323
346
|
def _doctor_memory_shortlist(memory_dir: str | None = None) -> list:
|
|
324
|
-
"""(id, body, ts) for
|
|
347
|
+
"""(id, body, (ts, event_at)) for active permanent T2 records, same two clocks as T1."""
|
|
325
348
|
return store.permanent_records(memory_dir)
|
|
326
349
|
|
|
327
350
|
|
|
@@ -329,7 +352,8 @@ def _memory_records(memory_dir: str | None = None) -> dict:
|
|
|
329
352
|
"""id -> (meta, body) for active permanent records, the lookup the adjudicator needs."""
|
|
330
353
|
out = {}
|
|
331
354
|
for cid, body, ts in store.permanent_records(memory_dir):
|
|
332
|
-
|
|
355
|
+
ts, event_at = ts if isinstance(ts, (tuple, list)) else (ts, "")
|
|
356
|
+
out[cid] = ({"id": cid, "ts": ts, "event_at": event_at}, body)
|
|
333
357
|
return out
|
|
334
358
|
|
|
335
359
|
|
|
@@ -491,6 +515,23 @@ def _age_label(ts: str):
|
|
|
491
515
|
return "%dd ago" % int(hours // 24)
|
|
492
516
|
|
|
493
517
|
|
|
518
|
+
def _world_age_label(ts: str, event_at: str):
|
|
519
|
+
"""Age of the state a record describes, in days, when that differs from `ts`.
|
|
520
|
+
|
|
521
|
+
Two clocks, and the gap between them is the point: `ts` says when this store first
|
|
522
|
+
held the claim, `event_at` says when the record itself says the thing was true. A
|
|
523
|
+
fact read out of a six-month-old document yesterday carries both, and a model told
|
|
524
|
+
only "recorded 1d ago" reads it as current. Returns None when the two agree to the
|
|
525
|
+
day, because saying the same thing twice is noise, and None on an unparseable value
|
|
526
|
+
rather than a guessed number.
|
|
527
|
+
"""
|
|
528
|
+
a = store._hours_since(ts) if ts else None
|
|
529
|
+
b = store._hours_since(event_at) if event_at else None
|
|
530
|
+
if a is None or b is None or int(a // 24) == int(b // 24):
|
|
531
|
+
return None
|
|
532
|
+
return int(b // 24)
|
|
533
|
+
|
|
534
|
+
|
|
494
535
|
def _inject_rank(records: list, query: str) -> list:
|
|
495
536
|
"""Order records by overlap with ``query``, newest first as the tie-break.
|
|
496
537
|
|
|
@@ -570,6 +611,12 @@ def cmd_inject(a) -> int:
|
|
|
570
611
|
age = _age_label(c.get("ts", ""))
|
|
571
612
|
if age:
|
|
572
613
|
line += " (recorded %s)" % age
|
|
614
|
+
# Separately, and only when it differs: how old the state the record describes
|
|
615
|
+
# is. "recorded 1d ago, true 180d ago" is a late note about an old world, and a
|
|
616
|
+
# reader that only sees the first half takes it for current.
|
|
617
|
+
world = _world_age_label(c.get("ts", ""), c.get("event_at", ""))
|
|
618
|
+
if world is not None:
|
|
619
|
+
line += " (described a state %dd old)" % world
|
|
573
620
|
recs.append(line)
|
|
574
621
|
served.append(c.get("id", ""))
|
|
575
622
|
if not recs:
|
package/sleepkit/consolidate.py
CHANGED
|
@@ -24,7 +24,14 @@ def pending_dir() -> str:
|
|
|
24
24
|
return d
|
|
25
25
|
|
|
26
26
|
|
|
27
|
-
|
|
27
|
+
# The window one consolidation pass is shown. It is a display limit, not a policy:
|
|
28
|
+
# whatever falls below it is NOT read by the pass and NOT pruned, and until 0.8.18
|
|
29
|
+
# nothing said so. BUFFER_WINDOW is now named in one place so the count of records
|
|
30
|
+
# left outside the window can be reported instead of being inferred from len().
|
|
31
|
+
BUFFER_WINDOW = 120
|
|
32
|
+
|
|
33
|
+
|
|
34
|
+
def _fmt_buffer(records: list[dict], limit: int = BUFFER_WINDOW) -> str:
|
|
28
35
|
lines = []
|
|
29
36
|
for r in records[-limit:]:
|
|
30
37
|
lines.append("- [%s|%s] %s" % (r.get("kind", "fact"), r.get("source", "?"),
|
|
@@ -32,6 +39,41 @@ def _fmt_buffer(records: list[dict], limit: int = 120) -> str:
|
|
|
32
39
|
return "\n".join(lines)
|
|
33
40
|
|
|
34
41
|
|
|
42
|
+
def _window_split(records: list) -> tuple:
|
|
43
|
+
"""(sent, skipped): records inside the format window and those left outside it.
|
|
44
|
+
|
|
45
|
+
Only the newest BUFFER_WINDOW records are shown to the model, so a record with an
|
|
46
|
+
older ts is never read by any pass and never pruned (prune only runs on records the
|
|
47
|
+
pass consumed). One distinct session arriving with more than a window's worth of
|
|
48
|
+
captures would therefore be silently dropped from consolidation. The caller reports
|
|
49
|
+
the skipped count so that possibility is visible instead of invisible.
|
|
50
|
+
"""
|
|
51
|
+
if len(records) <= BUFFER_WINDOW:
|
|
52
|
+
return list(records), []
|
|
53
|
+
return list(records[-BUFFER_WINDOW:]), list(records[:-BUFFER_WINDOW])
|
|
54
|
+
|
|
55
|
+
|
|
56
|
+
def _warn_field(skipped: int, total: int) -> dict:
|
|
57
|
+
"""{"warn": sentence} when records were left out, {} when none were.
|
|
58
|
+
|
|
59
|
+
A key that is always present but sometimes empty makes every caller test the value;
|
|
60
|
+
absent-when-there-is-nothing-to-say keeps `"warn" not in result` meaningful.
|
|
61
|
+
"""
|
|
62
|
+
return {"warn": _window_warning(skipped, total)} if skipped > 0 else {}
|
|
63
|
+
|
|
64
|
+
|
|
65
|
+
def _window_warning(skipped: int, total: int) -> str:
|
|
66
|
+
"""One sentence for both paths that can leave records outside the window.
|
|
67
|
+
|
|
68
|
+
Shared so `--dry-run` and `--sleep` cannot drift: a preview that says nothing while
|
|
69
|
+
the real pass warns is worse than no preview.
|
|
70
|
+
"""
|
|
71
|
+
if skipped <= 0:
|
|
72
|
+
return ""
|
|
73
|
+
return ("%d of %d buffer records fell outside the %d-record window and were "
|
|
74
|
+
"not read or pruned" % (skipped, total, BUFFER_WINDOW))
|
|
75
|
+
|
|
76
|
+
|
|
35
77
|
def existing_pool() -> str:
|
|
36
78
|
lines = []
|
|
37
79
|
for c in store.list_candidates(with_body=True):
|
|
@@ -146,8 +188,11 @@ def distill(session: str | None = None, model: str | None = None,
|
|
|
146
188
|
return {"ok": True, "reason": "buffer empty", "proposals": 0, "accepted": 0,
|
|
147
189
|
"consumed": False}
|
|
148
190
|
projects, unaddressed = _project_groups(recs)
|
|
191
|
+
_sent, outside_window = _window_split(recs)
|
|
149
192
|
if dry_run:
|
|
150
193
|
return {"ok": True, "dry_run": True, "buffer_records": len(recs),
|
|
194
|
+
"outside_window": len(outside_window),
|
|
195
|
+
**_warn_field(len(outside_window), len(recs)),
|
|
151
196
|
"projects": [o for o, _ in projects], "unaddressed": len(unaddressed),
|
|
152
197
|
"would_send": _fmt_buffer(recs)[:2000]}
|
|
153
198
|
if not projects:
|
|
@@ -183,6 +228,11 @@ def distill(session: str | None = None, model: str | None = None,
|
|
|
183
228
|
ing["ok"] = False
|
|
184
229
|
ing["buffer_records"] = len(recs)
|
|
185
230
|
ing["unaddressed"] = len(unaddressed)
|
|
231
|
+
# Records the pass was never shown. Not an error and not a bug on its own; the point
|
|
232
|
+
# is that a night which leaves any behind says so, because "proposals: 2" on a
|
|
233
|
+
# 300-record buffer reads as a complete pass and is not one.
|
|
234
|
+
ing["outside_window"] = len(outside_window)
|
|
235
|
+
ing.update(_warn_field(len(outside_window), len(recs)))
|
|
186
236
|
# "consumed" is what entitles a sleep to prune: the pass produced proposals.
|
|
187
237
|
# Unaddressed records were never read by any pass, so they must not be pruned.
|
|
188
238
|
return ing
|
|
@@ -427,7 +477,8 @@ def run_sleep(session=None, model=None, budget_usd=1.0, min_records=3,
|
|
|
427
477
|
state["last_sleep"] = paths.now()
|
|
428
478
|
state["sleep_count"] = int(state.get("sleep_count", 0)) + 1
|
|
429
479
|
state["last_sleep_result"] = {k: result.get(k) for k in
|
|
430
|
-
("proposals", "accepted", "refused", "llm_exit"
|
|
480
|
+
("proposals", "accepted", "refused", "llm_exit",
|
|
481
|
+
"buffer_records", "outside_window")}
|
|
431
482
|
paths.write_state(state)
|
|
432
483
|
# Synaptic pruning only after a consolidation that actually produced
|
|
433
484
|
# proposals. Pruning a buffer the model never read would silently discard
|
package/sleepkit/store.py
CHANGED
|
@@ -720,7 +720,12 @@ def permanent_records(memory_dir: str | None = None) -> list:
|
|
|
720
720
|
if meta.get("status") == "retracted":
|
|
721
721
|
continue
|
|
722
722
|
ref = str(meta.get("id") or name[:-3]).strip()
|
|
723
|
-
|
|
723
|
+
# The recorded claim time, not the promotion date: a memory written down long after
|
|
724
|
+
# the fact is a late note about an old state. Carried as (claim_time, event_at) so
|
|
725
|
+
# _older_newer can read the same two clocks the T1 shortlist carries.
|
|
726
|
+
out.append((ref, body or "",
|
|
727
|
+
(str(meta.get("ts") or meta.get("promoted_at") or ""),
|
|
728
|
+
str(meta.get("event_at") or ""))))
|
|
724
729
|
return out
|
|
725
730
|
|
|
726
731
|
|
|
@@ -921,12 +926,43 @@ def candidate_pairs(recs: list, limit: int = CONTRADICTION_MAX_PAIRS) -> list:
|
|
|
921
926
|
return [(o, n) for _s, o, n in scored[:limit]]
|
|
922
927
|
|
|
923
928
|
|
|
929
|
+
def _clock(entry) -> tuple[str, str]:
|
|
930
|
+
"""Accept either the old bare-ts map value or this module's (ts, event_at) pair."""
|
|
931
|
+
if isinstance(entry, (tuple, list)):
|
|
932
|
+
ts = str(entry[0] or "") if len(entry) > 0 else ""
|
|
933
|
+
ev = str(entry[1] or "") if len(entry) > 1 else ""
|
|
934
|
+
return ts, ev
|
|
935
|
+
return str(entry or ""), ""
|
|
936
|
+
|
|
937
|
+
|
|
924
938
|
def _older_newer(a: str, b: str, ts_by_id: dict) -> tuple[str, str]:
|
|
925
|
-
"""Order two candidate ids by
|
|
926
|
-
|
|
927
|
-
|
|
928
|
-
|
|
929
|
-
|
|
939
|
+
"""Order two candidate ids by the claim's own time, oldest first.
|
|
940
|
+
|
|
941
|
+
Which side is the claim and which is the correction is a statement about the world,
|
|
942
|
+
not about this store, so the ordering reads both clocks the record carries:
|
|
943
|
+
|
|
944
|
+
- ts is when the store first held the claim, event_at is when the record says the
|
|
945
|
+
described thing was true. A session can file an old fact today, so ts alone
|
|
946
|
+
would name a document quote recorded this morning as "the newer claim" and hand
|
|
947
|
+
back a supersede command pointing the wrong way.
|
|
948
|
+
- event_at wins whenever the pair can be compared on it at all, for BOTH sides.
|
|
949
|
+
Mixing the clocks (one side's ts against the other's event_at) compares two
|
|
950
|
+
different axes and just relocates the bug: the documented late note carries both,
|
|
951
|
+
so its ts would still be what the comparison read.
|
|
952
|
+
- With no stated date on either side, or on only one, the capture ts pair decides
|
|
953
|
+
and, when those tie or are missing, the id does. A pair is always returned, so
|
|
954
|
+
`supersede <older> <newer>` is always a runnable command.
|
|
955
|
+
|
|
956
|
+
Values may be a bare timestamp (older callers) or a (ts, event_at) pair.
|
|
957
|
+
"""
|
|
958
|
+
ta, ea = _clock(ts_by_id.get(a, ""))
|
|
959
|
+
tb, eb = _clock(ts_by_id.get(b, ""))
|
|
960
|
+
# One axis per comparison: event_at when both records state one, capture ts otherwise.
|
|
961
|
+
ca, cb = (ea, eb) if (ea and eb) else (ta, tb)
|
|
962
|
+
if ca and cb and ca != cb:
|
|
963
|
+
return (a, b) if ca < cb else (b, a)
|
|
964
|
+
if ca != cb:
|
|
965
|
+
return (a, b) if not ca else (b, a)
|
|
930
966
|
return (a, b) if a <= b else (b, a)
|
|
931
967
|
|
|
932
968
|
|
|
@@ -974,7 +1010,12 @@ def doctor(memory_dir: str | None = None, targets: list[str] | None = None,
|
|
|
974
1010
|
"value": needle})
|
|
975
1011
|
pool_records = list_candidates(with_body=True)
|
|
976
1012
|
pool = [(c["id"], c.get("_body", "")) for c in pool_records]
|
|
977
|
-
|
|
1013
|
+
# Two clocks per record: ts is when this store first held the claim, event_at is when
|
|
1014
|
+
# the record itself says the described thing was true. A session can learn an old fact
|
|
1015
|
+
# today (ts new, event_at old), so "which side is the older claim" is a world-time
|
|
1016
|
+
# question and cannot be answered from ts alone. The pair is (ts, event_at) and the
|
|
1017
|
+
# ordering rule lives in _older_newer.
|
|
1018
|
+
pool_ts = {c["id"]: (str(c.get("ts", "")), str(c.get("event_at", ""))) for c in pool_records}
|
|
978
1019
|
for i in range(len(pool)):
|
|
979
1020
|
for j in range(i + 1, len(pool)):
|
|
980
1021
|
sim = similarity(pool[i][1], pool[j][1])
|
|
@@ -990,6 +1031,8 @@ def doctor(memory_dir: str | None = None, targets: list[str] | None = None,
|
|
|
990
1031
|
if mi != mj:
|
|
991
1032
|
# Name the older/newer side so a person does not diff two timestamps by
|
|
992
1033
|
# eye: the older record is the claim, the newer one is the correction.
|
|
1034
|
+
# "Older" is world time (_older_newer), so a record learned late about an
|
|
1035
|
+
# old state is still named as the claim and not as the fix.
|
|
993
1036
|
# The suggestion is a command to RUN, never one this code runs -
|
|
994
1037
|
# superseding on a keyword match could kill a true memory, so the gate
|
|
995
1038
|
# stays human.
|
|
@@ -1029,6 +1072,18 @@ def doctor(memory_dir: str | None = None, targets: list[str] | None = None,
|
|
|
1029
1072
|
out.append({"issue": "unused-candidate-never-served", "id": c["id"],
|
|
1030
1073
|
"age_days": int(age_h // 24),
|
|
1031
1074
|
"injected": 0, "rediscovered": 0})
|
|
1075
|
+
# The one input limit that is not a content judgement: consolidation is shown only
|
|
1076
|
+
# the newest WINDOW records, so a buffer larger than that carries records no pass
|
|
1077
|
+
# will read and no prune will clear. Measured on the honest definition of a pass -
|
|
1078
|
+
# what the CLI actually hands over - and reported here rather than raised to a
|
|
1079
|
+
# failure, because a large buffer may be a real backlog rather than an error.
|
|
1080
|
+
from . import consolidate as _consolidate
|
|
1081
|
+
t0_records = buffer_counts()["records"]
|
|
1082
|
+
if t0_records > _consolidate.BUFFER_WINDOW:
|
|
1083
|
+
out.append({"issue": "buffer-exceeds-consolidation-window",
|
|
1084
|
+
"buffer_records": t0_records,
|
|
1085
|
+
"window": _consolidate.BUFFER_WINDOW,
|
|
1086
|
+
"outside_window": t0_records - _consolidate.BUFFER_WINDOW})
|
|
1032
1087
|
for cid, body in pool:
|
|
1033
1088
|
for path, text in bodies.items():
|
|
1034
1089
|
# MEMORY.md is the index: it restates every entry by design, so scoring
|