pi-sleep-agent 0.8.17 → 0.8.18

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -27,6 +27,11 @@ Sleep adds a reviewable memory cycle to nine selectable coding-agent targets. It
27
27
 
28
28
  - **Automatic capture.** Hooks and the Pi extension collect useful observations while you work.
29
29
  - **Budgeted consolidation.** Sleep runs only after the record and time gates pass.
30
+ - **A named consolidation window.** One pass is shown the newest 120 T0 records. A buffer
31
+ past that is never silent: `distill --dry-run` and the pass itself report how many records
32
+ fell outside the window, `status` says it before the next pass runs, and `doctor` reports
33
+ it as its own finding. Records outside the window are shown to no pass and cleared by no
34
+ prune, which is why the number is printed instead of being inferred from the proposal count.
30
35
  - **Review before permanence.** New memories stay provisional until you promote or refuse them.
31
36
  - **Nightly dream cycle.** Mine repeated preferences, consolidate recent observations, and measure memory effects while you are away.
32
37
  - **Measured replay.** Run the same task with memory off and on before deciding whether a memory helps. Each completed pass leaves a one-line quality summary (`helps / hurts / ties`) in `status` and in the nightly report, so the question stays answerable between passes; a pass whose two arms ran the same prompt says it measured nothing instead of reporting zeros.
@@ -336,7 +341,9 @@ Run the two dream components separately when you want more control:
336
341
  | `./sleep.py status -v` | ... and the last measured memory quality (`3 helps / 1 hurts / 4 ties from 8 samples`, or "no replay measurement has run yet") |
337
342
  | `./sleep.py mine --dry-run` | Find preferences repeated across sessions |
338
343
  | `./sleep.py nightly --dry-run` | Preview the unattended nightly workflow |
339
- | `./sleep.py doctor` | Find duplicate or overlapping memory records, flag contradicting candidates (same subject, one side carries a correction marker) and print the `supersede <older> <newer>` command to run, flag provisional candidates not rediscovered for 4+ weeks, flag records no session was ever served for 8+ weeks, flag supersede pointers that name a record no longer in the store, and report records parked in quarantine awaiting a human (report only: it never supersedes, drops, merges or un-parks anything itself) |
344
+ | `./sleep.py doctor` | Find duplicate or overlapping memory records, flag contradicting candidates (same subject, one side carries a correction marker) and print the `supersede <older> <newer>` command to run, flag provisional candidates not rediscovered for 4+ weeks, flag records no session was ever served for 8+ weeks, flag supersede pointers that name a record no longer in the store, report a T0 buffer larger
345
+ than the 120-record consolidation window (with the count left outside it), and report records
346
+ parked in quarantine awaiting a human (report only: it never supersedes, drops, merges or un-parks anything itself) |
340
347
  | `./sleep.py doctor --stale <value>` | Show which approved records still hold a stale value (e.g. a changed URL) |
341
348
  | `./sleep.py doctor --contradictions` | Ask a model to adjudicate shortlisted pairs of provisional records for conflicts (`--budget`, default $0.25); report-only, and each finding carries the `supersede <older> <newer>` command to run |
342
349
  | `./sleep.py doctor --contradictions --tier t2` | Adjudicate pairs of **approved** permanent memory instead of candidates (`--tier both` does both); a T2 finding is labelled `contradicting-memory` and carries the `demote <older>` command, because approved memory is un-approved, never superseded |
package/docs/EVIDENCE.md CHANGED
@@ -496,3 +496,38 @@ crossing a process boundary with a prompt-recording fake model),
496
496
  Not changed, and named here so the leftover is not mistaken for coverage: `candidate_pairs()`
497
497
  still orders by `ts` and still makes one model call per pair, and `cmd_doctor` does not call it.
498
498
  It is a duplicate of the ordering rule with a per-pair cost, left alone deliberately.
499
+
500
+
501
+ ## 2026-11-03: the consolidation window is named, and the record left outside it is counted
502
+
503
+ `_fmt_buffer` showed a consolidation pass only the newest 120 T0 records. Everything older was
504
+ never read by any pass, and because pruning only clears what a pass consumed, never cleared either.
505
+ Nothing said so: the night reported `proposals: 2` whether the buffer held 12 records or 300, and
506
+ that number reads as a complete pass.
507
+
508
+ Measured before changing anything, and the honest reading is that the cap had never bitten here:
509
+ `~/.sleep/archive/buffer` holds 31 consumed buffers totalling 259 records, the largest a single
510
+ night ever held was 73, and the live buffer held 1. So this was a latent silent loss, not an
511
+ observed one, and the change is about the report rather than about the limit.
512
+
513
+ What the window now costs to see, all three surfaces checked against a real 127-record buffer in a
514
+ throwaway `SLEEP_HOME`:
515
+
516
+ | surface | without the window | with records outside it |
517
+ |---|---|---|
518
+ | `distill --dry-run` / the pass result | key absent | `outside_window: 7`, `warn: "7 of 127 buffer records fell outside the 120-record window and were not read or pruned"` |
519
+ | `sleep.py status` | T0 line unchanged | `T0 buffer 127 records ... [next pass reads the newest 120; 7 outside the window]` |
520
+ | `sleep.py doctor` | no finding | `{"issue": "buffer-exceeds-consolidation-window", "buffer_records": 127, "window": 120, "outside_window": 7}` |
521
+
522
+ `run_sleep` now persists `buffer_records` and `outside_window` in `state.json` next to `proposals`,
523
+ so a night that left records behind is readable after the fact. The window lives in one place,
524
+ `sleepkit.consolidate.BUFFER_WINDOW`, and the sentence is built by one helper (`_window_warning`,
525
+ wrapped by `_warn_field` so the key is absent rather than empty when there is nothing to report) so
526
+ the preview and the real pass cannot drift. Pinned by `tests/test_cli.py::BufferWindow` (5 tests,
527
+ crossing a process boundary): under the window nothing appears in any of the three surfaces, at
528
+ `WINDOW + 7` the count and the sentence say both numbers, and a pass the record/time gate *refuses*
529
+ still reports the buffer size it did not read.
530
+
531
+ Deliberately NOT changed, since the two are separable and only the reporting was asked for: the pass
532
+ still reads only the newest window and the rest still waits for the next night. Sending a large
533
+ buffer in several passes is a separate change with its own cost (one model call per slice).
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "pi-sleep-agent",
3
- "version": "0.8.17",
3
+ "version": "0.8.18",
4
4
  "description": "A reviewable sleep and dream cycle for Pi agent memory.",
5
5
  "type": "module",
6
6
  "license": "MIT",
package/sleepkit/cli.py CHANGED
@@ -70,7 +70,14 @@ def cmd_status(a) -> int:
70
70
  _p(replay_mod.quality_line())
71
71
  except Exception:
72
72
  pass
73
- _p("T0 buffer %d records / %d files / %d bytes (captured while you worked)" % (bc["records"], bc["files"], bc["bytes"]))
73
+ line = "T0 buffer %d records / %d files / %d bytes (captured while you worked)" % (bc["records"], bc["files"], bc["bytes"])
74
+ # A window smaller than the buffer is the one case where the next pass cannot read
75
+ # everything it is standing on. Say it up front instead of leaving it to be inferred
76
+ # from the proposals count of a night that already happened.
77
+ if bc["records"] > consolidate.BUFFER_WINDOW:
78
+ line += " [next pass reads the newest %d; %d outside the window]" % (
79
+ consolidate.BUFFER_WINDOW, bc["records"] - consolidate.BUFFER_WINDOW)
80
+ _p(line)
74
81
  _p("T1 candidates %d provisional, %d refused, %d total (distilled, shown as unverified context)" % (len(prov), len(ref), len(cands)))
75
82
  _p(" quarantine %d parked (needs human eyes) (held out until you review)" % len(q))
76
83
  retracted = []
@@ -24,7 +24,14 @@ def pending_dir() -> str:
24
24
  return d
25
25
 
26
26
 
27
- def _fmt_buffer(records: list[dict], limit: int = 120) -> str:
27
+ # The window one consolidation pass is shown. It is a display limit, not a policy:
28
+ # whatever falls below it is NOT read by the pass and NOT pruned, and until 0.8.18
29
+ # nothing said so. BUFFER_WINDOW is now named in one place so the count of records
30
+ # left outside the window can be reported instead of being inferred from len().
31
+ BUFFER_WINDOW = 120
32
+
33
+
34
+ def _fmt_buffer(records: list[dict], limit: int = BUFFER_WINDOW) -> str:
28
35
  lines = []
29
36
  for r in records[-limit:]:
30
37
  lines.append("- [%s|%s] %s" % (r.get("kind", "fact"), r.get("source", "?"),
@@ -32,6 +39,41 @@ def _fmt_buffer(records: list[dict], limit: int = 120) -> str:
32
39
  return "\n".join(lines)
33
40
 
34
41
 
42
+ def _window_split(records: list) -> tuple:
43
+ """(sent, skipped): records inside the format window and those left outside it.
44
+
45
+ Only the newest BUFFER_WINDOW records are shown to the model, so a record with an
46
+ older ts is never read by any pass and never pruned (prune only runs on records the
47
+ pass consumed). One distinct session arriving with more than a window's worth of
48
+ captures would therefore be silently dropped from consolidation. The caller reports
49
+ the skipped count so that possibility is visible instead of invisible.
50
+ """
51
+ if len(records) <= BUFFER_WINDOW:
52
+ return list(records), []
53
+ return list(records[-BUFFER_WINDOW:]), list(records[:-BUFFER_WINDOW])
54
+
55
+
56
+ def _warn_field(skipped: int, total: int) -> dict:
57
+ """{"warn": sentence} when records were left out, {} when none were.
58
+
59
+ A key that is always present but sometimes empty makes every caller test the value;
60
+ absent-when-there-is-nothing-to-say keeps `"warn" not in result` meaningful.
61
+ """
62
+ return {"warn": _window_warning(skipped, total)} if skipped > 0 else {}
63
+
64
+
65
+ def _window_warning(skipped: int, total: int) -> str:
66
+ """One sentence for both paths that can leave records outside the window.
67
+
68
+ Shared so `--dry-run` and `--sleep` cannot drift: a preview that says nothing while
69
+ the real pass warns is worse than no preview.
70
+ """
71
+ if skipped <= 0:
72
+ return ""
73
+ return ("%d of %d buffer records fell outside the %d-record window and were "
74
+ "not read or pruned" % (skipped, total, BUFFER_WINDOW))
75
+
76
+
35
77
  def existing_pool() -> str:
36
78
  lines = []
37
79
  for c in store.list_candidates(with_body=True):
@@ -146,8 +188,11 @@ def distill(session: str | None = None, model: str | None = None,
146
188
  return {"ok": True, "reason": "buffer empty", "proposals": 0, "accepted": 0,
147
189
  "consumed": False}
148
190
  projects, unaddressed = _project_groups(recs)
191
+ _sent, outside_window = _window_split(recs)
149
192
  if dry_run:
150
193
  return {"ok": True, "dry_run": True, "buffer_records": len(recs),
194
+ "outside_window": len(outside_window),
195
+ **_warn_field(len(outside_window), len(recs)),
151
196
  "projects": [o for o, _ in projects], "unaddressed": len(unaddressed),
152
197
  "would_send": _fmt_buffer(recs)[:2000]}
153
198
  if not projects:
@@ -183,6 +228,11 @@ def distill(session: str | None = None, model: str | None = None,
183
228
  ing["ok"] = False
184
229
  ing["buffer_records"] = len(recs)
185
230
  ing["unaddressed"] = len(unaddressed)
231
+ # Records the pass was never shown. Not an error and not a bug on its own; the point
232
+ # is that a night which leaves any behind says so, because "proposals: 2" on a
233
+ # 300-record buffer reads as a complete pass and is not one.
234
+ ing["outside_window"] = len(outside_window)
235
+ ing.update(_warn_field(len(outside_window), len(recs)))
186
236
  # "consumed" is what entitles a sleep to prune: the pass produced proposals.
187
237
  # Unaddressed records were never read by any pass, so they must not be pruned.
188
238
  return ing
@@ -427,7 +477,8 @@ def run_sleep(session=None, model=None, budget_usd=1.0, min_records=3,
427
477
  state["last_sleep"] = paths.now()
428
478
  state["sleep_count"] = int(state.get("sleep_count", 0)) + 1
429
479
  state["last_sleep_result"] = {k: result.get(k) for k in
430
- ("proposals", "accepted", "refused", "llm_exit")}
480
+ ("proposals", "accepted", "refused", "llm_exit",
481
+ "buffer_records", "outside_window")}
431
482
  paths.write_state(state)
432
483
  # Synaptic pruning only after a consolidation that actually produced
433
484
  # proposals. Pruning a buffer the model never read would silently discard
package/sleepkit/store.py CHANGED
@@ -1072,6 +1072,18 @@ def doctor(memory_dir: str | None = None, targets: list[str] | None = None,
1072
1072
  out.append({"issue": "unused-candidate-never-served", "id": c["id"],
1073
1073
  "age_days": int(age_h // 24),
1074
1074
  "injected": 0, "rediscovered": 0})
1075
+ # The one input limit that is not a content judgement: consolidation is shown only
1076
+ # the newest WINDOW records, so a buffer larger than that carries records no pass
1077
+ # will read and no prune will clear. Measured on the honest definition of a pass -
1078
+ # what the CLI actually hands over - and reported here rather than raised to a
1079
+ # failure, because a large buffer may be a real backlog rather than an error.
1080
+ from . import consolidate as _consolidate
1081
+ t0_records = buffer_counts()["records"]
1082
+ if t0_records > _consolidate.BUFFER_WINDOW:
1083
+ out.append({"issue": "buffer-exceeds-consolidation-window",
1084
+ "buffer_records": t0_records,
1085
+ "window": _consolidate.BUFFER_WINDOW,
1086
+ "outside_window": t0_records - _consolidate.BUFFER_WINDOW})
1075
1087
  for cid, body in pool:
1076
1088
  for path, text in bodies.items():
1077
1089
  # MEMORY.md is the index: it restates every entry by design, so scoring