@rubytech/create-realagent 1.0.832 → 1.0.834

package/payload/platform/plugins/memory/skills/conversation-archive/SKILL.md

@@ -69,7 +69,10 @@ Capture that path. The progress file lives at `data/accounts/$ACCOUNT_ID/logs/co
 Optional flags:
 - `--timezone <iana>` — IANA zone for timestamps (default `Europe/London`).
 - `--date-format <DD/MM/YY|MM/DD/YY|DD/MM/YYYY|MM/DD/YYYY>` — WhatsApp only; override auto-detect for ambiguous locales.
-- `--session-gap-hours <N>` — gap threshold (in hours) used to split parsed messages into sessions for chunking (default `12`).
+
+Sessions split deterministically at 8h gap (fixed code constant — Task 902). The pre-902 `--session-gap-hours` flag is removed: chunked chat-mode classify (Task 902 sub-scope C) absorbs oversize sessions internally so the gap value is no longer an operational lever. Passing the legacy flag FAILs at `phase=argv`.
+
+`--rebuild` (operator-issued only; never agent-autonomous — see Doctrine below): treats the run as first-ingest, deletes prior `:Section:Conversation` chunks for THIS export's `archiveSha256`, and re-classifies the entire archive. The skipped-cursor-lookup means a `--rebuild` and a fresh-export-with-different-bytes are operationally equivalent — one is destructive on identifiable bytes, the other is additive.
 
 ### Heartbeat protocol
 
@@ -79,9 +82,35 @@ Every 30 s while the background bash is live:
 tail -n 100 <progress-file>
 ```
 
-Surface fresh `[conversation-archive] classify-session sessionIndex=K/N ...` lines as chat tokens — one chat line per fresh progress line. The SDK silence timer resets on every output token; a 30 s poll cadence keeps the silence wall (180 s) untripped on a steady classify loop.
+The bin emits one **derived** progress line per session boundary; the agent surfaces those (and only those) as chat tokens.
+
+Grep contract — surface ONLY these line shapes to operator chat:
+
+| Line shape (bin emits) | Surface as |
+|---|---|
+| `[conversation-archive] progress sessionIndex=K/N pct=P chunks-so-far=X elapsed-ms=M` | `Classifying session K/N (P%) — chunks so far X — elapsed <Ms>` |
+| `[conversation-archive] WARN cleanup-dropped chunks=N archiveSha256=<sha>` | **WARN** `cleanup-dropped chunks=N — prior data deleted (only legitimate under --rebuild; STOP and verify if not)` |
+| `[conversation-archive] FAIL phase=…` | verbatim + yield (do not retry) |
+
+Suppress all other `[conversation-archive] …` lines (raw `classify-session`, `session-committed`, `cleanup-skipped`, etc.) — they remain on disk for diagnosis but are not operator-stream noise. The bin's `progress` line is computed deterministically (Task 902 sub-scope F): per-tick percentage arithmetic and running totals come from one place, not from agent prose-rule arithmetic which drifts.
+
+If the tail returns no NEW `progress` line for 60 s while the bg pid is still live: surface one `still on session K/N (P%) — last update <Δ>s ago` and stop emitting until the next advance. Forbidden: emitting "polling progress file" or any meta-status the operator did not request. Forbidden: blind-reissue of the bash.
+
+### Bash poll `description` field — pin progress forward (Task 905)
+
+Each heartbeat `tail` call's `description` carries the most recent progress signal forward, parsed from the prior poll's last surfaced line. Chat timeline rows collapse to the `description`; the body holding the actual progress is hidden until the operator expands every row, so the progress belongs in the title.
 
-If the tail returns no NEW lines for 60 s while the bg pid is still live: surface the last progress line verbatim to the operator, and stop polling until they direct otherwise. **Do not blind-reissue the bash.**
+Format: `Heartbeat poll — sessionIndex=K/N pct=P chunks-so-far=X`
+
+The first poll (no prior progress yet) is `Heartbeat poll — starting`. Every subsequent poll updates the suffix from the last surfaced progress line; if the most recent poll surfaced the `still on session …` stall line, the suffix becomes `Heartbeat poll — sessionIndex=K/N pct=P stalled <Δ>s`.
+
+The literal label `Heartbeat poll` never shortens — not to `Poll N`, not to `polling`, not to anything else. Forbidden: any heartbeat `description` that drops the progress suffix once a `progress` line has been surfaced; any label other than `Heartbeat poll`.
+
+### Subagent dispatch contract (Task 905)
+
+When the heartbeat loop is delegated to a subagent (ad-hoc `Agent` invocation watching a `run_in_background` bash for this skill), the dispatch prompt MUST include the **Heartbeat protocol** subsection above verbatim — the grep contract, the description-format clause, and the no-NEW-progress branch. Subagents do not inherit memory or per-turn feedback; the dispatch prompt is the only carrier of this convention across the subagent boundary.
+
+A subagent that emits `Poll 58`, `Poll 69`, or any non-canonical label is a contract violation, not stylistic drift. The main agent's prompt-construction step is the gate: the subagent prompt is wrong if the literal string `Heartbeat poll —` is missing from it.
 
 ### Resume after silent stall
 
@@ -93,11 +122,12 @@ Forbidden: blind-reissue without reading the progress file first.
 - Picks the normaliser for `--source`. WhatsApp: locates `_chat.txt` (zip / dir / direct file), parses deterministically, computes `archiveSha256`. Other sources interpret the path according to their own format.
 - Validates every distinct parsed senderName against the closed set of `{owner, participants...}` candidate names. Any miss LOUD-FAILs `parser-miss reason="senderName=<verbatim> not in confirmed participant set ..."`.
 - Computes `conversationIdentity` from accountId + sorted participant elementIds.
-- Looks up any prior `:ConversationArchive` carrying that identity → reads `lastIngestedMessageHash`. If found, slices parsed lines after the cursor (delta-append). Cursor not found → `FAIL delta-cursor-missing`. Cursor at last line → empty-delta noop (exit 0, no writes).
-- Sessionizes the delta lines at the operator-supplied gap-hours boundary.
-- For each session: renders as turn-attributed text (`[ts] Sender: body\n…`) and calls `memory-classify` with `mode='chat'`. Returns one or more `:Section:Conversation` chunk specs.
+- **Without `--rebuild`** (default): looks up any prior `:ConversationArchive` carrying that identity → reads `lastIngestedMessageHash`. If found, slices parsed lines after the cursor (delta-append). Cursor not found → `FAIL delta-cursor-missing`. Cursor at last line → empty-delta noop (exit 0, no writes; existing chunk count surfaced in JSON summary). **With `--rebuild`** (Task 902 sub-scope B): skips the prior-cursor lookup entirely; every line is treated as first-ingest delta; the first session's `memory-ingest` deletes prior chunks for THIS `archiveSha256` and the new cursor overwrites the stale one in the same MERGE.
+- Sessionizes the delta lines at the fixed 8h gap (Task 902 sub-scope D — code constant, no flag).
+- Computes the stable archive `title` once (Task 902 sub-scope A): `<source> · <owner> ↔ <others> · <YYYY-MM-DD>→<YYYY-MM-DD>`.
+- For each session: renders as turn-attributed text (`[ts] Sender: body\n…`) and calls `memory-classify` with `mode='chat'`. Oversize sessions dispatch to the chunked-chat path (Task 902 sub-scope C) — sessionize size is no longer an operator concern. Returns one or more `:Section:Conversation` chunk specs.
 - **Per-session checkpoint** (Task 900 sub-scope E): immediately after each successful classify, calls `memory-ingest` for THAT session's chunks AND advances `lastIngestedMessageHash` / `lastIngestedMessageAt` on the parent `:ConversationArchive` to the last message of that session — atomic (chunk writes + cursor advance happen inside one Cypher transaction). A kill mid-loop leaves the cursor at session N-1's last message; re-issuing with the same argv and `--session-id` resumes from session N onward without re-classifying prior sessions.
-- Server MERGEs the parent on `conversationIdentity`, MERGEs `:PARTICIPANT_IN` edges, CREATEs new chunks, extends the `:NEXT` chain from its tail. The cleanup-by-`archiveSha256` step (drops THIS export's prior chunks) runs only on the FIRST per-session call of a run; subsequent sessions skip cleanup or they would delete chunks just written. Every node and edge stamps `source=<enum>` and `createdByAgent='conversation-archive'`.
+- Server MERGEs the parent on `conversationIdentity` (writing `title` ON CREATE and COALESCE-on-MATCH so a stable title is never overwritten), MERGEs `:PARTICIPANT_IN` edges, CREATEs new chunks, extends the `:NEXT` chain from its tail. The cleanup-by-`archiveSha256` step runs ONLY when `--rebuild` is set AND only on the FIRST per-session call of a run. Every node and edge stamps `source=<enum>` and `createdByAgent='conversation-archive'`.
 
 NO insight pass runs. Phase 2 (operator-driven `:Observation` / `:Task` / `:Preference` derivation against chunks) is its own follow-up task — and applies uniformly to every source once chunks exist.
 
@@ -143,8 +173,15 @@ For an empty-delta re-import (`delta.kind === "empty-delta"`): emit only message
 
 ## Idempotency
 
-- **Re-running the same export bytes** is a no-op: the cleanup-by-`archiveSha256` step drops THIS export's prior chunks and re-creates them with identical content. `:NEXT` chain length unchanged.
+- **Re-running the same export bytes is a no-op by default** (Task 902 sub-scope B). The bin reports the existing chunk count and exits 0; cursor and chunks are unchanged. The pre-902 contract that ran cleanup-by-`archiveSha256` on every same-bytes re-run was the destructive primitive that combined with `--session-gap-hours` to silently destroy 138 chunks (Adam Mackay incident, 2026-05-04).
 - **Re-running with appended messages** (a fresh export from the same chat with new messages at the tail): cursor lookup finds the prior `lastIngestedMessageHash`, slices new messages, sessionizes only those, and appends new chunks at the tail of the existing `:NEXT` chain. Pre-existing chunks are never touched.
+- **Destructive rebuild** requires the explicit `--rebuild` flag — and only the operator may pass it (see Doctrine below). With `--rebuild`, the bin treats the run as first-ingest, the FIRST session's writer cleans prior chunks for the matching `archiveSha256`, the cursor is overwritten by that session's MERGE, and the heartbeat surfaces a WARN line so the destructive action is operator-visible.
+- **Re-ingesting into a previously-trashed archive** revives the parent before MERGE. If the operator drag-trashed a `:ConversationArchive` and then re-runs ingest (delta or `--rebuild`), `memory-ingest`'s `ingestConversationArchive` strips `:Trashed` from the matching `conversationIdentity` so the MERGE binds to a non-trashed parent — otherwise the chunks would land under a `:Trashed` parent the graph UI hides, making the entire archive (and every freshly-written chunk under it) invisible until something else clears the label. Mirrors the KnowledgeDocument trash-revival path (Task 576).
+
+## Doctrine
+
+- **`--rebuild` is operator-issued only.** The agent must NEVER propose `--rebuild` as a fix for any FAIL phase, classifier error, oversize-session report, or partial-state recovery. The operator must type `--rebuild` themselves. Agent-autonomous `--rebuild` would re-introduce the silent-partial-wipe surface that Task 902 closed; the flag exists exclusively for operators reasoning about identifiable export bytes they want to re-classify.
+- **Use a fresh export** (different `archiveSha256`) for every legitimate re-classify case the agent surfaces. Same-bytes re-classify is the operator's call, not the agent's.
 
 ## Verification (post-write)
 

package/payload/platform/scripts/lib/resolve-account-dir.sh

@@ -8,7 +8,9 @@
 #
 # Contract (inputs via env vars):
 #   ACCOUNTS_DIR — {installDir}/data/accounts
-#   USERS_FILE   — {installDir}/platform/config/users.json
+#   USERS_FILE   — $HOME/<brand.configDir>/users.json (Task 904 — was
+#                  {installDir}/platform/config/users.json pre-Task-904, but
+#                  that path lived in the installer's wipe zone)
 #
 # Contract (outputs via env vars):
 #   ACCOUNT_ID   — resolved uuid (existing kept, or freshly minted)

package/payload/platform/scripts/migrate-import.sh

@@ -376,7 +376,9 @@ if [ -f "$PINS_FILE" ]; then
       # users.json from .admin-pin, but only at install time — and install runs
       # before this script writes .admin-pin. Self-contained migration writes
       # users.json directly, mirroring seed-neo4j.sh's migration branch.
-      USERS_FILE="$INSTALL_DIR/platform/config/users.json"
+      # Task 904 — write to the persistent location (outside the install wipe
+      # zone) so subsequent installs do not overwrite this row with a stale backup.
+      USERS_FILE="$PIN_DIR/users.json"
       if [ ! -f "$USERS_FILE" ]; then
         USER_ID="$(cat /proc/sys/kernel/random/uuid 2>/dev/null || python3 -c 'import uuid; print(uuid.uuid4())')"
         PIN_HASH="$(cat "$PIN_DIR/.admin-pin")"

package/payload/platform/scripts/seed-neo4j.sh

@@ -50,7 +50,15 @@ CYPHER_SHELL="cypher-shell"
 # no identity match aborts loud; .trash/ is operator-emptied, never auto-cleaned.
 # shellcheck source=lib/resolve-account-dir.sh
 . "$SCRIPT_DIR/lib/resolve-account-dir.sh"
-USERS_FILE="$PROJECT_DIR/config/users.json" resolve_and_sweep_account_dir
+# Resolve brand-aware persistent users.json path before the resolver runs.
+# Mirrors the resolution lower in this file (around the post-Task-904 USERS_FILE
+# block); duplicated here because resolve_and_sweep_account_dir runs first.
+_RESOLVER_CONFIG_DIR_NAME=".maxy"
+if [ -f "$PROJECT_DIR/config/brand.json" ]; then
+  _RESOLVER_BRAND_CFG_DIR=$(python3 -c "import json; print(json.load(open('$PROJECT_DIR/config/brand.json')).get('configDir',''))" 2>/dev/null || true)
+  [ -n "$_RESOLVER_BRAND_CFG_DIR" ] && _RESOLVER_CONFIG_DIR_NAME="$_RESOLVER_BRAND_CFG_DIR"
+fi
+USERS_FILE="$HOME/$_RESOLVER_CONFIG_DIR_NAME/users.json" resolve_and_sweep_account_dir
 
 mkdir -p "$ACCOUNT_DIR/agents/admin" "$ACCOUNT_DIR/.claude" "$ACCOUNT_DIR/specialists/.claude-plugin" "$ACCOUNT_DIR/specialists/agents"
 
@@ -376,9 +384,8 @@ echo "  Account $ACCOUNT_ID at $ACCOUNT_DIR"
 # ------------------------------------------------------------------
 
 CONFIG_DIR="$PROJECT_DIR/config"
-USERS_FILE="$CONFIG_DIR/users.json"
 
-# Resolve the brand-specific config directory for .admin-pin location.
+# Resolve the brand-specific config directory.
 # Mirrors the logic in platform/ui/app/lib/paths.ts.
 _CONFIG_DIR_NAME=".maxy"
 if [ -f "$CONFIG_DIR/brand.json" ]; then
@@ -386,6 +393,9 @@ if [ -f "$CONFIG_DIR/brand.json" ]; then
   [ -n "$_BRAND_CFG_DIR" ] && _CONFIG_DIR_NAME="$_BRAND_CFG_DIR"
 fi
 ADMIN_PIN_FILE="$HOME/$_CONFIG_DIR_NAME/.admin-pin"
+# Task 904 — users.json lives under $HOME/$_CONFIG_DIR_NAME, not the wipe zone.
+USERS_FILE="$HOME/$_CONFIG_DIR_NAME/users.json"
+mkdir -p "$HOME/$_CONFIG_DIR_NAME"
 
 # Only create users.json if it doesn't exist AND .admin-pin exists (migration case).
 # Fresh installs: users.json is created by set-pin POST during onboarding.