@rubytech/create-maxy 1.0.830 → 1.0.832

package/payload/platform/plugins/memory/skills/conversation-archive/SKILL.md

@@ -40,32 +40,64 @@ The flow:
 
 DM and group follow the identical flow. A 1:1 chat resolves 2 senders; a group resolves N senders. Identity (`conversationIdentity = sha256(accountId + ":" + sortedParticipantElementIds.join(","))`) is identical regardless of group size — DM and group use the same MERGE key. Source format (whatsapp / telegram / slack) does not affect identity.
 
-## Step 2 — invoke the script
+## Step 2 — invoke the script (background + heartbeat)
 
-Single Bash call:
+The Haiku classify loop on a multi-thousand-message archive can take minutes. The Claude Agent SDK closes the response stream after **180 seconds of model silence**, so issuing the bash in the foreground guarantees a stall. The script is invoked in the **background** and its progress file is polled on a heartbeat — the script writes one fsync'd progress line per Haiku iteration, and surfacing those lines as chat tokens keeps the SDK silence timer reset.
+
+### Identity (Task 900)
+
+`ACCOUNT_ID` and `USER_ID` come from the agent's process env (plumbed by `spawn-env.ts` into every Bash subprocess). The script LOUD-FAILs on missing or malformed values; the operator never passes them as flags. The owner `:AdminUser` elementId is derived from `(ACCOUNT_ID, USER_ID)` inside the script via Cypher — `--owner-element-id` is gone.
+
+### Background invocation
 
 ```bash
 bash platform/plugins/memory/bin/conversation-archive-ingest.sh <source-path> \
   --source <whatsapp|telegram|signal|linkedin-messages|zoom-transcript|meeting-minutes|imessage|slack|other> \
-  --owner-element-id <id> \
   --participant-person-ids <id1>,<id2>,... \
-  --scope <admin|public>
+  --scope <admin|public> \
+  --session-id <unique-id>
 ```
 
+Issue with `run_in_background: true`. The script's first stdout line is:
+
+```
+[conversation-archive] progress-file=<absolute-path-to-progress-file>
+```
+
+Capture that path. The progress file lives at `data/accounts/$ACCOUNT_ID/logs/conversation-archive-$SESSION_ID.log` and is opened with append + fsync per write, so every line is durable on disk before the script proceeds.
+
 Optional flags:
-- `--account-id <id>` — explicit account id when more than one exists under `data/accounts/` (Phase 0 has one).
 - `--timezone <iana>` — IANA zone for timestamps (default `Europe/London`).
 - `--date-format <DD/MM/YY|MM/DD/YY|DD/MM/YYYY|MM/DD/YYYY>` — WhatsApp only; override auto-detect for ambiguous locales.
-- `--session-gap-hours <N>` — gap threshold (in hours) used to split parsed messages into sessions for chunking (default `12`). Smaller values produce more sessions, more Haiku calls, finer chunks; larger values group more messages per session.
+- `--session-gap-hours <N>` — gap threshold (in hours) used to split parsed messages into sessions for chunking (default `12`).
+
+### Heartbeat protocol
+
+Every 30 s while the background bash is live:
+
+```bash
+tail -n 100 <progress-file>
+```
+
+Surface fresh `[conversation-archive] classify-session sessionIndex=K/N ...` lines as chat tokens — one chat line per fresh progress line. The SDK silence timer resets on every output token; a 30 s poll cadence keeps the silence wall (180 s) untripped on a steady classify loop.
+
+If the tail returns no NEW lines for 60 s while the bg pid is still live: surface the last progress line verbatim to the operator, and stop polling until they direct otherwise. **Do not blind-reissue the bash.**
+
+### Resume after silent stall
+
+If the SDK injects `"A background task (subagent) didn't finish"` or the `"180 seconds of silence"` banner, the **first action** is `tail -n 200 <progress-file>` for the prior `sessionId`. The progress file's last line names the session number, message count, and phase — that's the resume context. Every per-session commit is atomic (chunks + cursor advance happen in one Cypher transaction via `memory-ingest`'s `MERGE`/`CREATE`); a kill mid-loop leaves the cursor at session N-1's last message. Re-issuing the same bash with the same argv and same `--session-id` resumes from session N onward without re-classifying anything already written.
+
+Forbidden: blind-reissue without reading the progress file first.
 
-The script:
+### What the script does
 - Picks the normaliser for `--source`. WhatsApp: locates `_chat.txt` (zip / dir / direct file), parses deterministically, computes `archiveSha256`. Other sources interpret the path according to their own format.
 - Validates every distinct parsed senderName against the closed set of `{owner, participants...}` candidate names. Any miss LOUD-FAILs `parser-miss reason="senderName=<verbatim> not in confirmed participant set ..."`.
 - Computes `conversationIdentity` from accountId + sorted participant elementIds.
 - Looks up any prior `:ConversationArchive` carrying that identity → reads `lastIngestedMessageHash`. If found, slices parsed lines after the cursor (delta-append). Cursor not found → `FAIL delta-cursor-missing`. Cursor at last line → empty-delta noop (exit 0, no writes).
 - Sessionizes the delta lines at the operator-supplied gap-hours boundary.
-- For each session: renders as turn-attributed text (`[ts] Sender: body\n…`) and calls `memory-classify` with `mode='chat'` (the classifier sees turn-attributed text only — never the source format). Returns one or more `:Section:Conversation` chunk specs.
-- Calls `memory-ingest` with `parentLabel='ConversationArchive'` and `source=<enum>`. Server MERGEs the parent on `conversationIdentity`, MERGEs `:PARTICIPANT_IN` edges from each confirmed participant, drops any prior chunks stamped with this `archiveSha256` (idempotency for re-running the same export bytes), CREATEs new chunks, extends the existing `:NEXT` chain from its tail, advances `lastIngestedMessageHash` + `lastIngestedMessageAt`. Every node and edge stamps `source=<enum>` and `createdByAgent='conversation-archive'`.
+- For each session: renders as turn-attributed text (`[ts] Sender: body\n…`) and calls `memory-classify` with `mode='chat'`. Returns one or more `:Section:Conversation` chunk specs.
+- **Per-session checkpoint** (Task 900 sub-scope E): immediately after each successful classify, calls `memory-ingest` for THAT session's chunks AND advances `lastIngestedMessageHash` / `lastIngestedMessageAt` on the parent `:ConversationArchive` to the last message of that session — atomic (chunk writes + cursor advance happen inside one Cypher transaction). A kill mid-loop leaves the cursor at session N-1's last message; re-issuing with the same argv and `--session-id` resumes from session N onward without re-classifying prior sessions.
+- Server MERGEs the parent on `conversationIdentity`, MERGEs `:PARTICIPANT_IN` edges, CREATEs new chunks, extends the `:NEXT` chain from its tail. The cleanup-by-`archiveSha256` step (drops THIS export's prior chunks) runs only on the FIRST per-session call of a run; subsequent sessions skip cleanup or they would delete chunks just written. Every node and edge stamps `source=<enum>` and `createdByAgent='conversation-archive'`.
 
 NO insight pass runs. Phase 2 (operator-driven `:Observation` / `:Task` / `:Preference` derivation against chunks) is its own follow-up task — and applies uniformly to every source once chunks exist.
 

package/payload/platform/scripts/lib/resolve-account-dir.sh

@@ -78,8 +78,26 @@ PY
 
   local n=${#candidates[@]}
 
-  # Case 0: no candidates — fresh install, mint new UUID
+  # Case 0: no candidates — fresh install, mint new UUID.
+  # Task 900 sub-scope F — atomic-mint discipline: refuse to mint a new
+  # accountId if any stub dir (subdir present, account.json missing) exists
+  # under data/accounts/. The Task 900 root cause was install 1 minting
+  # accountId X then losing X/account.json (cron stamped X/logs/ first); the
+  # next install saw zero candidates and minted Y, orphaning X/logs/. The
+  # guard turns that silent regression into an operator-actionable abort.
   if [ "$n" -eq 0 ]; then
+    local stub_dirs=()
+    for dir in "$ACCOUNTS_DIR"/*/; do
+      [ -d "$dir" ] || continue
+      base="$(basename "$dir")"
+      case "$base" in .*) continue ;; esac
+      [ -f "$dir/account.json" ] && continue
+      stub_dirs+=("$base")
+    done
+    if [ "${#stub_dirs[@]}" -gt 0 ]; then
+      echo "==> [seed] FAIL phase=mint reason=\"stub-account-dirs present (no account.json): ${stub_dirs[*]}\" — refusing to mint a fresh accountId; remove or repair the stub before re-running install" >&2
+      return 1
+    fi
     ACCOUNT_ID="$(cat /proc/sys/kernel/random/uuid 2>/dev/null || python3 -c 'import uuid; print(uuid.uuid4())')"
     ACCOUNT_DIR="$ACCOUNTS_DIR/$ACCOUNT_ID"
     return 0