@rubytech/create-maxy 1.0.757 → 1.0.759

package/payload/platform/plugins/whatsapp-import/skills/whatsapp-import/SKILL.md

@@ -0,0 +1,122 @@
+---
+name: whatsapp-import
+description: Import a WhatsApp `_chat.txt` export into a {{productName}} Neo4j graph as a Conversation with chronologically-chained Messages, then derive typed insights (mentions, preferences, commitments, observed relationships) as first-class graph entities. Triggers when the user asks to import a WhatsApp chat, ingest a `_chat.txt` file, or drops the contents of an "Export Chat" folder into chat. Distinct from the live `whatsapp` plugin (Baileys); this is import-from-export only.
+---
+
+# WhatsApp Import
+
+Ingests a WhatsApp "Export Chat" archive — `_chat.txt` plus media attachments — into a {{productName}} Neo4j graph. Two passes:
+
+1. **Deterministic ingest** — Conversation + Messages + chronology + sender edges, written via the fixed Cypher inside `memory-archive-write`.
+2. **Insight extraction** — analysis-derived nodes and edges (mentions, topics, preferences, commitments, observed relationships) written via existing `memory-write` / `memory-update` tools after pass 1 completes.
+
+Every node and edge carries `source='whatsapp'`, `createdByAgent='whatsapp-import'`, `createdBySession=<this-skill-run-uuid>`, and `archiveSourceFile=<sha256-prefix>` so the operator can grep this ingest's footprint at any time.
+
+## Owner + participant confirmation (mandatory first step)
+
+A WhatsApp export belongs to exactly one operator (the person whose phone produced the export) and contains messages from a known set of senders. Both must be confirmed before any line is written. The flow:
+
+### Step 1 — Owner
+
+The owner is metadata: who exported this chat. Stamped on the `:Conversation` node as `createdBySession` provenance. The owner is **not** a row-level subject — every message has its own sender.
+
+1. List every `:AdminUser` in the graph: `{userId, name, accountId(s)}`.
+2. Ask the operator: "Who exported this `_chat.txt`?" Accept either an existing `:AdminUser` userId or a new external `:Person` (with `givenName`+`familyName`+ at least one of `email`/`telephone`).
+3. Echo the chosen owner back verbatim. Require explicit yes/no confirmation before proceeding.
+4. Persist the resolved owner's `elementId` as `$ownerNodeId`.
+
+### Step 2 — Participants
+
+Parse the `_chat.txt` per [export-parse.md](references/export-parse.md). For each distinct sender name, capture: `{senderName, firstSeen, lastSeen, messageCount}`. Display the list in chat with these counts; the operator sees who they're about to ingest before any write.
+
+For each distinct sender, ask the operator to choose:
+
+- **Existing `:AdminUser`** — typically themselves (when their own messages are in the export). Resolve via `memory-search` by `userId` or `name`. Persist the elementId.
+- **Existing `:Person`** — match by `givenName`+`familyName`, `email`, or `telephone`. Use `memory-search` to find candidates; if multiple match, surface them and require operator pick. Persist the elementId.
+- **New external `:Person`** — mint via `memory-write` with `givenName`+`familyName`+ at least one of `email`/`telephone`. Provenance: `source='whatsapp'`, `createdByAgent='whatsapp-import'`, `createdBySession=$sessionId`. Capture the resulting elementId.
+- **Skip** — exclude this sender's messages from the import. Operator may pick this for noisy auto-replies, bots, etc.
+
+**Refusing to invent identity is load-bearing.** The skill never silently mints a `:Person` from a WhatsApp display name alone (which is often just a phone number or "Mum"). A new `:Person` requires confirmation of `givenName`+`familyName`+ contact information. This is the first contract `feedback_archives_are_not_documents.md` enforces.
+
+### Step 3 — Persist the participant map
+
+Build `$participantNodeIds = {senderName → senderElementId}`. Echo back to operator one final time (`Confirm: 5 senders, 4 :AdminUser/:Person, 1 skipped — proceed?`). On yes, the participant map flows into every row of the `memory-archive-write` call.
+
+### Step 4 — Same-person, multiple display-names heuristic
+
+WhatsApp displays a sender by their phone-saved name when known, by phone number otherwise. If the operator's contact list changed mid-conversation, the same person may appear under two distinct senderNames (`+44 7...` and `Joel Smalley`). Detect this heuristically: surface any senderName that is digit-only (a phone number) and ask `Is "+44 7..." the same person as "Joel Smalley"?`. On yes, both senderNames map to the same elementId. On no, keep them distinct.
+
+## Selective-ingest threshold (bulk archives)
+
+WhatsApp 1:1 chats commonly contain 1,000–10,000 messages; group chats 10,000+. Writing all of them in one shot defeats compression-on-write and produces a landfill graph. The skill compresses by interrogating the operator before the bulk write.
+
+**Threshold:** when the parsed `rows[]` count exceeds **100 messages**, pause and ask the operator to filter along the natural axes:
+
+- **Date range** — "messages between 2026-01-01 and 2026-04-01"
+- **Sender** — "only messages from Joel and Sarah"
+- **Keyword** — body contains "Q3 report" / "office hours" / etc.
+- **All** — accept the full archive (rare; only for small chats or when the operator explicitly wants every message)
+
+Apply the chosen filter to `rows[]` before invoking `memory-archive-write`. Compress on write, never after — a 5,000-message blanket import is noise; a 200-message filtered import is signal.
+
+When the threshold trips, emit one log line BEFORE the prompt:
+
+```
+[whatsapp-import] selective-ingest-gate count=<n> threshold=100 axes=date,sender,keyword
+```
+
+## Stable IDs
+
+- `conversationId = whatsapp-export:<sha256(_chat.txt bytes)>:<accountId>` — same archive, same operator account → idempotent re-import. Different archive (even for the same conversation) → different conversationId.
+- `messageId = whatsapp-export:<conversationId>:<lineHash>` where `lineHash = sha256(<original-line-text>)`. Re-imports of the same archive are zero-write idempotent; re-exports with appended messages add the delta cleanly.
+
+## Timezone
+
+WhatsApp's `[DD/MM/YY, HH:MM:SS]` line prefix lacks a timezone offset. The skill **must not silently assume UTC.** When the timezone is non-obvious (the operator hasn't said where they were when the messages were sent), ask:
+
+> The export uses `[DD/MM/YY, HH:MM:SS]` but doesn't include a timezone. Which timezone should I tag these messages with? (e.g., Europe/London, America/New_York, UTC)
+
+Convert each parsed timestamp to ISO 8601 with the supplied offset before passing to `memory-archive-write`. The Cypher's `datetime()` then preserves the exact instant.
+
+## Execution model
+
+1. **Parse** — Read `_chat.txt` per [export-parse.md](references/export-parse.md). Build the parsed-line structure: `{senderName, dateSent, body, sequenceIndex}`. Skip system messages and media-only lines, increment counters.
+2. **Owner+participant confirmation** — Steps 1–3 above. Persist `$ownerNodeId` + `$participantNodeIds`.
+3. **Selective-ingest gate** — If `parsedLines.length > 100`, pause for filter selection. Apply filter.
+4. **Build rows[]** — Map each parsed line to `{messageId, conversationId, senderNodeId, senderName, dateSent (ISO 8601), body, sequenceIndex}`. Compute `messageId` per line.
+5. **Build conversation block** — `{conversationId, archiveSourceFile, firstMessageAt, lastMessageAt, participantCount, messageCount}` from the rows[].
+6. **Dispatch** `mcp__memory__memory-archive-write` once with `archiveType='whatsapp-export'`, `ownerNodeId`, `conversation`, `participantNodeIds` (the distinct elementIds from the map), `rows`, `sessionId`. The tool MERGEs the Conversation, MERGEs Messages, links PART_OF + SENT + PARTICIPANT_IN edges per row, and runs the `finalize` hook to MERGE the NEXT chronology by dateSent ordering.
+7. **Emit per-export log line:**
+   ```
+   [whatsapp-import] file=<chat.txt> conversationId=<cid> participants=<n> messages-parsed=<n> media-skipped=<n> system-skipped=<n> ms=<elapsed>
+   ```
+8. **Insight pass** — Run pass 2 per [insight-extraction.md](references/insight-extraction.md). Read the just-written messages via `memory-search`, classify within the specialist's own LLM turn, and write typed observations through `memory-write` / `memory-update`. Emit:
+   ```
+   [whatsapp-import] insight-pass model=sonnet chunks=<n> mentions=<n> preferences=<n> tasks=<n> observed-relationships=<n> novel-insights=<n> ms=<elapsed>
+   ```
+
+## Doctrine — raw Cypher and `cypher-shell` are forbidden in this skill
+
+All writes route through `mcp__memory__memory-archive-write` (bulk Conversation+Messages) or `mcp__memory__memory-write` / `mcp__memory__memory-update` (second-pass typed observations). The agent never authors Cypher. If the operator hits a write shape these tools do not express, **do not improvise** — surface the gap as a structured task per the database-operator's LOUD-FAIL prerogative. See [database-operator.md](../../../../templates/specialists/agents/database-operator.md#prerogatives).
+
+## LOUD-FAIL on parse errors
+
+If [export-parse.md](references/export-parse.md)'s grammar doesn't match a line (genuine parser failure, not a documented skip case), emit:
+
+```
+[whatsapp-import] parse-error file=<chat.txt> line=<n> reason=<r>
+```
+
+…and abort the import. Do NOT silently truncate or guess. The operator gets a named error; we keep no half-truths in the graph.
+
+## Idempotency contract
+
+Re-importing the same `_chat.txt` is a no-op (`createdMessages=0`, `mergedMessages=N`, NEXT chain unchanged). Re-importing a re-exported file with appended messages adds only the delta and extends the NEXT chain to cover the new tail. Both paths are server-enforced via MERGE on `messageId` and the finalize hook's idempotent NEXT-MERGE.
+
+## Verification (post-write)
+
+- `MATCH (c:Conversation:WhatsAppConversation {conversationId: $cid}) RETURN c.messageCount, c.participantCount, c.firstMessageAt, c.lastMessageAt` — agrees with the per-export log line counts.
+- `MATCH (m:Message:WhatsAppMessage)-[:PART_OF]->(c {conversationId: $cid}) RETURN count(m)` — equals post-filter line count.
+- `MATCH p=(m:Message {conversationId: $cid})-[:NEXT*]->(end) WITH max(length(p)) AS chain RETURN chain` — equals `messageCount - 1`.
+- `MATCH (m:Message {conversationId: $cid}) RETURN min(m.dateSent), max(m.dateSent)` — matches the file's first/last lines (modulo the operator-confirmed timezone).
+- `MATCH (n) WHERE n.createdBySession = $sessionId RETURN labels(n) AS l, count(*) ORDER BY count(*) DESC` — the full graph footprint of this ingest, sortable by label.

package/payload/platform/plugins/whatsapp-import/skills/whatsapp-import/references/conversation-and-messages.md

@@ -0,0 +1,99 @@
+# Reference: Conversation + Messages row schema
+
+This reference defines the `archiveType='whatsapp-export'` row + conversation block + participantNodeIds shapes that the [SKILL.md](../SKILL.md) passes to `mcp__memory__memory-archive-write`. The Cypher body is fixed server-side in [`platform/plugins/memory/mcp/src/tools/memory-archive-write.ts`](../../../../memory/mcp/src/tools/memory-archive-write.ts); the agent's responsibility ends at producing valid input.
+
+## Tool input shape
+
+```json
+{
+  "archiveType": "whatsapp-export",
+  "ownerNodeId": "<elementId of :AdminUser or :Person — operator who exported the chat>",
+  "sessionId": "<UUID — generated once per skill run>",
+  "conversation": {
+    "conversationId": "whatsapp-export:<sha256(file)>:<accountId>",
+    "archiveSourceFile": "<sha256(file)>",
+    "firstMessageAt": "2026-03-14T10:15:23Z",
+    "lastMessageAt": "2026-04-21T18:42:11Z",
+    "participantCount": 2,
+    "messageCount": 174
+  },
+  "participantNodeIds": ["<elemId-Joel>", "<elemId-Sarah>"],
+  "rows": [
+    {
+      "messageId": "whatsapp-export:<conversationId>:<lineHash>",
+      "conversationId": "<same as conversation.conversationId>",
+      "senderNodeId": "<elemId — operator-confirmed>",
+      "senderName": "Joel",
+      "dateSent": "2026-03-14T10:15:23Z",
+      "body": "Quick question about the deck —\ndo you have the v3 PDF anywhere?",
+      "sequenceIndex": 0
+    },
+    ...
+  ]
+}
+```
+
+## What the server does (informational, not the agent's responsibility)
+
+Each batch (max 500 rows) runs in one `executeWrite` transaction:
+
+1. **MERGE Conversation** — `(:Conversation:WhatsAppConversation {conversationId})`. ON CREATE stamps full provenance + scalar metadata (firstMessageAt, lastMessageAt, participantCount, messageCount, archiveSourceFile). ON MATCH refreshes the moving scalars (`lastMessageAt`, `messageCount`, `participantCount`) plus `lastImportedAt` / `lastImportedBySession` for re-import audit; createdBy* stamps stay frozen.
+2. **MATCH sender by elementId** — `WHERE elementId(sender) = row.senderNodeId`. The verifyParticipants preflight already confirmed every senderNodeId resolves; this MATCH never fails at runtime.
+3. **MERGE Message** — `(:Message:WhatsAppMessage {messageId})`. ON CREATE stamps provenance, body, dateSent, senderName (raw, audit-only), sequenceIndex.
+4. **MERGE PART_OF** — `(m)-[:PART_OF]->(c)`.
+5. **MERGE SENT** — `(sender)-[:SENT]->(m)`. Per-row, idempotent.
+6. **MERGE PARTICIPANT_IN** — `(sender)-[:PARTICIPANT_IN]->(c)`. Per-row, but rolls up across rows because MERGE on the same edge between the same pair is idempotent — a 200-message conversation with 2 participants ends with 2 PARTICIPANT_IN edges, not 200.
+
+After all batches, the **finalize** hook runs one additional `executeWrite` transaction:
+
+7. **MERGE NEXT chain** — `MATCH (m:Message)-[:PART_OF]->(c {conversationId})` ordered by `dateSent ASC, sequenceIndex ASC`, then UNWIND consecutive pairs, MERGE `(prev)-[:NEXT]->(next)`. Idempotent on MERGE — re-imports add only the new tail edges. Returns `nextEdges` count for the per-call log line.
+
+## Natural keys + provenance
+
+| Entity | Key | Provenance fields stamped |
+|---|---|---|
+| `:Conversation:WhatsAppConversation` | `conversationId` | `accountId, source='whatsapp', createdByAgent='whatsapp-import', createdBySource='whatsapp-import', createdBySession, createdAt=datetime(), scope='admin', agentType='admin', archiveSourceFile`. ON re-import, also: `lastImportedAt, lastImportedBySession`. |
+| `:Message:WhatsAppMessage` | `messageId` | Same provenance set + `conversationId, dateSent, body, senderName, sequenceIndex, scope='admin'`. |
+| `[:SENT]` | (sender, message) pair | `source='whatsapp', createdAt=datetime()`. |
+| `[:PARTICIPANT_IN]` | (sender, conversation) pair | `source='whatsapp', createdAt=datetime()`. |
+| `[:PART_OF]` | (message, conversation) pair | (no edge properties needed; the edge type is the signal). |
+| `[:NEXT]` | (prev message, next message) pair | `source='whatsapp', createdAt=datetime()`. |
+
+## Edges this reference does NOT mint
+
+- `(:AdminUser)-[:OWNS]->(:Conversation)` — explicitly NOT created. The exporter is metadata-on-Conversation provenance, not a graph relationship. A `:Conversation` is owned by its `accountId`, the same way other account-scoped nodes are. Adding an OWNS edge would duplicate property-based scope with edge-based scope (anti-pattern; see `feedback_account_scope_is_a_property.md`).
+- `(:Conversation)-[:OCCURRED_AT]->(:Date)` — date is a scalar property on the Conversation, not a separate node. Querying by date uses the property, not an edge traversal.
+- `(:Message)-[:HAS_BODY]->(:Text)` — the body is a property on the Message; no separate Text node.
+
+If a future query pattern reveals one of these is genuinely useful, file a separate task — never add them ad-hoc here.
+
+## Divergence from live-channel `persistMessage`
+
+The live `whatsapp` plugin's `persistMessage` in [`platform/ui/app/lib/neo4j-store.ts`](../../../../../ui/app/lib/neo4j-store.ts) holds a per-conversation lock when chaining NEXT — concurrent inbound messages to the same conversation would otherwise race to insert NEXT pointers. Bulk import does **not** need that lock because:
+
+1. The whole import runs in one `database-operator` turn, no concurrency.
+2. The finalize step runs ONE Cypher statement that scopes to a single `conversationId` and rebuilds the chain in `dateSent ASC` order. MERGE is idempotent; concurrent re-runs (which can't happen in one turn) would converge regardless.
+
+If a future change runs imports in parallel across conversations, that's still safe — each conversationId's NEXT chain is rebuilt independently. If two parallel imports targeted the same conversationId (operator scripting error), MERGE-on-NEXT idempotency saves the graph; the latest write wins on edge properties.
+
+## Counter shape returned to the agent
+
+```json
+{
+  "archiveType": "whatsapp-export",
+  "processedRows": 174,
+  "counters": {
+    "createdMessages": 174,
+    "relationshipsCreated": 524,
+    "labelsAdded": 348,
+    "nextEdgesProcessed": 173,
+    "nextEdgesCreated": 173
+  },
+  "errors": [],
+  "nextChainStatus": "ok"
+}
+```
+
+The skill displays the counts to the operator in chat and emits the `[whatsapp-import] file=...` log line using `processedRows`, `nextEdgesCreated`, and the elapsed time. The exact counter keys evolve (per-handler), so the skill should iterate the `counters` map rather than destructure specific names.
+
+When a batch errors, `errors[]` contains `{rowIndex, reason}`; `nextChainStatus` becomes `"partial"` (finalize still runs over whatever messages did land); the skill's chat output names the failed offset and the operator decides whether to re-import.

package/payload/platform/plugins/whatsapp-import/skills/whatsapp-import/references/export-parse.md

@@ -0,0 +1,102 @@
+# Reference: `_chat.txt` parsing
+
+WhatsApp's "Export Chat" produces a UTF-8 text file with a deterministic line grammar. This reference is the contract for converting that file into the parsed-line structure the [SKILL.md](../SKILL.md) builds rows from.
+
+## File-open invariants
+
+1. **UTF-8 only.** Open the file with explicit UTF-8 decoding. On encoding error, abort the import with a named LOUD-FAIL — never silently substitute or corrupt bodies. WhatsApp's modern apps emit UTF-8 reliably; an encoding error usually means the operator manually edited the file with a tool that broke it. Surface the named error so they can re-export.
+2. **No size cap from the parser.** The parser handles arbitrarily large files; the [SKILL.md](../SKILL.md)'s 100-message selective-ingest gate is the operator-facing compression layer.
+3. **Compute `archiveSourceFile = sha256(file bytes)` first.** The hash drives `conversationId` and lets re-imports of the same archive land idempotently.
+
+## Line grammar
+
+Every message line begins with a square-bracketed timestamp prefix followed by `<Sender>: <body>`:
+
+```
+[DD/MM/YY, HH:MM:SS] <Sender>: <body>
+```
+
+- `DD/MM/YY` — two-digit day, month, two-digit year (WhatsApp default; some locales emit `MM/DD/YY` — operator confirms ambiguous orderings during the timezone prompt).
+- `HH:MM:SS` — 24-hour time. Older exports may emit `HH:MM` (no seconds); treat seconds as 0 when absent.
+- `<Sender>` — the sender's display name as WhatsApp showed it. May be a saved contact name, a phone number with country code (`+44 7700 900123`), or "You" for messages sent by the operator (older exports).
+- `<body>` — message text, possibly multi-line.
+
+Trim trailing whitespace from each line before parsing.
+
+## Multi-line bodies
+
+A body that wraps to multiple lines continues onto subsequent lines that do **not** match the timestamp-prefix pattern. The parser must accumulate these continuation lines into the previous message's body, joining with `\n`. End-of-message is detected by the next timestamp prefix or end-of-file.
+
+```
+[14/03/26, 10:15:23] Joel: Quick question about the deck —
+do you have the v3 PDF anywhere?
+I checked Drive and only see v2.
+[14/03/26, 10:16:01] Sarah: Sec, will dig it out
+```
+
+Yields two messages:
+- `Joel: "Quick question about the deck —\ndo you have the v3 PDF anywhere?\nI checked Drive and only see v2."`
+- `Sarah: "Sec, will dig it out"`
+
+## System messages — skip with counter
+
+WhatsApp injects system-generated lines into the export for group events, contact changes, and security messages. These lines match the timestamp prefix but are **not** sent by a person and have no first-class graph value. Skip them at parse time, increment `systemSkipped`, and do not pass to `memory-archive-write`.
+
+Patterns to recognise (English; localisation expands the list):
+
+- `<Sender> created group "<name>"`
+- `<Sender> changed the subject from "<old>" to "<new>"`
+- `<Sender> changed this group's icon`
+- `<Sender> added <other>`
+- `<Sender> removed <other>`
+- `<Sender> left`
+- `<Sender>'s security code changed.`
+- `Messages and calls are end-to-end encrypted. ...` (the conversation header)
+- `You deleted this message.`
+- `This message was deleted.`
+
+Heuristic for the parser: if the body contains no spaces between `<Sender>` and a verb-phrase token, AND the body lacks a colon-after-sender separator, treat as a system message. Conservative — when uncertain, prefer to ingest as a message; the insight pass tolerates noisy bodies better than the parser would tolerate dropped real messages.
+
+## Media attachments — skip with counter
+
+Lines whose body indicates a media-only message (no text content) get skipped at parse time, increment `mediaSkipped`. Patterns:
+
+- `<Media omitted>` (when the operator chose "Without Media" on export)
+- `IMG-<digits>-<digits>.jpg (file attached)` / `.jpeg` / `.png` / `.heic` / `.gif`
+- `VID-<digits>-<digits>.mp4 (file attached)`
+- `PTT-<digits>-<digits>.opus (file attached)` (voice notes)
+- `AUD-<digits>-<digits>.opus (file attached)` (audio)
+- `STK-<digits>-<digits>.webp (file attached)` (stickers)
+- `<filename>.pdf (file attached)`
+- `<filename>.docx (file attached)`
+- `‎<...> attached: <filename>` (alternative format on some platforms)
+
+Mixed messages (text + media reference in one body) are kept as messages — only pure-media-only lines are skipped. The text body is retained.
+
+## Forwarded messages
+
+A forwarded message is prefixed with the invisible Unicode `‎` (U+200E LEFT-TO-RIGHT MARK) followed by metadata WhatsApp injects. Parse the body normally; the LRM character is preserved in `body` (the insight pass's classifier sees it as benign). Do not strip — the raw body's fidelity matters for downstream queries.
+
+## Edge cases
+
+- **Empty body** (timestamp prefix followed by sender colon but no text). Rare. Skip with `systemSkipped` increment — usually corresponds to a deleted message stub.
+- **Leading BOM** (U+FEFF at file start). Strip before parsing the first line.
+- **Mixed line endings** (`\r\n` vs `\n`). Normalise to `\n` before tokenisation.
+- **Sender containing a colon** (e.g., a contact named "Joel: Work"). The grammar splits on the FIRST `: ` (colon-space) after the timestamp prefix's closing `]`. Subsequent colons in the sender or body are preserved verbatim.
+
+## Parser output shape
+
+The parser returns `{conversationId, archiveSourceFile, parsedLines[], counters}` where:
+
+- `parsedLines[]: Array<{senderName: string, dateSent: string (ISO 8601 with operator-supplied timezone), body: string, sequenceIndex: number}>`
+- `counters: {parsed: n, systemSkipped: n, mediaSkipped: n, parseErrors: n}`
+
+The skill consumes this directly. The `messageId` is computed by the skill (not the parser) so the `lineHash` covers the original raw line, not the post-parse normalised body.
+
+## When to LOUD-FAIL
+
+- Encoding error at file open (UTF-8 decode fails partway).
+- Zero parsed lines after walking the file (the file isn't a `_chat.txt`).
+- A timestamp prefix matches but the body parse fails (no `: ` separator after the closing `]`) — emit `[whatsapp-import] parse-error file=<...> line=<n> reason=<r>` and abort.
+
+Never silently drop data the parser couldn't classify. The operator chooses to skip; the parser does not choose for them.

package/payload/platform/plugins/whatsapp-import/skills/whatsapp-import/references/insight-extraction.md

@@ -0,0 +1,118 @@
+# Reference: Insight extraction (pass 2)
+
+After the deterministic ingest completes, the [SKILL.md](../SKILL.md) runs a second pass that emits **first-class graph entities** capturing what the conversation is *about*. Insights are not summaries; they are typed nodes and edges the operator's queries can traverse.
+
+This pass runs INLINE in the database-operator specialist's own LLM turn — Sonnet — calling existing memory tools. There is no separate insight-extraction MCP tool. Reuse-over-invent: each observation maps onto an existing graph label or edge type wherever possible. `:Insight` is the last resort.
+
+## Reuse-over-invent — the six observation kinds
+
+| Kind | Maps to | Edge from | Edge to | Edge type | Notes |
+|---|---|---|---|---|---|
+| Third-party mention | `:Person` (existing) | `:Message` | `:Person` | `:MENTIONS` | Hallucination gate: see below. |
+| Recurring topic (≥3 mentions) | `:DefinedTerm` (existing) with `category:'topic'` | `:Conversation` | `:DefinedTerm` | `:DISCUSSES` (with `frequency` property) | Surface as a topic only if the term recurs ≥3 distinct messages. |
+| Expressed preference | `:Preference` (existing, see [schema.cypher:588](../../../../../neo4j/schema.cypher#L588-L595)) | `:Person` or `:AdminUser` | `:Preference` | `:HAS_PREFERENCE` + `(:Preference)-[:OBSERVED_IN]->(:Conversation)` | Preference text is a single observation, not a paragraph. |
+| Commitment / action item | `:Task` (existing) | `:Person` or `:AdminUser` | `:Task` | `:OWNS` | Set `dueDate` only when explicitly named ("by Friday", "next week"). Skip for vague "soon". |
+| Inter-person relationship | (matched `:Person` nodes) | `:Person` | `:Person` | `:RELATED_TO` (with `kind`, `evidenceMessageIds[]`) | Operator-confirmation gate before write — see below. |
+| Genuinely novel finding | `:Insight` (new label, last resort only) | `:Insight` | `:Message` | `:DERIVED_FROM` | Only when reuse-over-invent fails for every existing label. Self-rated `confidence` 0–1. |
+
+## Anti-hallucination gates
+
+The biggest risk in this pass is Sonnet writing edges to wrong-Person nodes. Two gates protect the graph:
+
+### Gate 1: `memory-search` BEFORE every `:MENTIONS` edge
+
+For every `:MENTIONS` edge candidate, the skill turn MUST run `memory-search(query=<mentioned-name>, kind='person')` first. The result determines what happens:
+
+- **0 hits** — the mentioned name doesn't exist in the graph. Skip silently. Do not auto-mint a `:Person` from a chat-mention alone.
+- **1 hit** — proceed if the match is unambiguous; surface to operator if ambiguous (see Gate 2).
+- **2+ hits** — ambiguous. Surface to operator before any edge writes.
+
+The agent never writes a `:MENTIONS` edge without the prior `memory-search`. This is a discipline gate: the inline classification is just `memory-search → memory-write`, never `memory-write` directly from raw classification.
+
+### Gate 2: First-name-only matches surface to operator regardless of hit count
+
+Single-first-name references in chat ("ask Sarah about Q3") have ambiguous referents even when `memory-search` returns one match — that one match might be the wrong Sarah. The rule:
+
+- The mention has a **disambiguator** (full name "Sarah Chen", phone, email, role context "Sarah at Acme") → `memory-search` 1-hit → write the edge.
+- The mention is a **first-name only** without disambiguator → ALWAYS surface to operator confirmation, regardless of `memory-search` result count.
+
+Surface format:
+
+```
+[whatsapp-import] mention-ambiguous name="Sarah" reason=first-name-only candidates=1 awaiting-operator-resolution
+```
+
+Followed by a chat prompt: `"Sarah" mentioned in message <messageId>. Found 1 :Person candidate: Sarah Chen (sarah@acme.com). Confirm? Yes / No / Pick another.`
+
+### Gate 3: `:RELATED_TO` between two existing distinct Persons
+
+When the second pass infers a relationship between two `:Person` nodes who both already exist in the graph (e.g., chat says "Joel and Sarah are working on Q3 together" → `(joel)-[:RELATED_TO {kind:'collaborator'}]->(sarah)`), surface to operator confirmation before write. The operator sees the inferred edge with both endpoints' names + the supporting message excerpts; on yes, the edge writes with `evidenceMessageIds: [...]`.
+
+The default for this gate is conservative — when in doubt, surface. False-positive RELATED_TO edges are graph noise; false-negative skips can be re-run.
+
+## Chunking strategy
+
+For conversations with 100+ messages, chunk the input to the inline LLM turn at ~50 messages per chunk. The classifier processes each chunk independently; the skill aggregates observations across chunks before writing. Aggregation deduplicates (the same `:MENTIONS` edge would otherwise be proposed once per chunk that referenced the same person).
+
+Per-chunk processing emits one log line for grep-ability:
+
+```
+[whatsapp-import] insight-pass-chunk index=<i> messages=<n> mentions-proposed=<n> preferences-proposed=<n> tasks-proposed=<n>
+```
+
+After all chunks complete and observations are written, the per-pass summary log line fires:
+
+```
+[whatsapp-import] insight-pass model=sonnet chunks=<n> mentions=<n> preferences=<n> tasks=<n> observed-relationships=<n> novel-insights=<n> ms=<elapsed>
+```
+
+The numbers in the summary line are **edges actually written**, not edges proposed. Proposed-but-skipped (zero hits, ambiguous-skipped, operator-rejected) don't count.
+
+## When to mint `:Insight`
+
+Only when **every** of the six observation kinds above fails to fit. Examples that DO fit existing labels:
+
+- "I prefer dark roast" → `:Preference`
+- "Send the deck Friday" → `:Task` with `dueDate`
+- "Sarah keeps mentioning Q3 reporting" → `:DefinedTerm{category:'topic', name:'Q3 reporting'}` + `:DISCUSSES`
+- "Joel works with Sarah" → `:RELATED_TO`
+
+Examples that genuinely warrant `:Insight`:
+
+- "The team's morale dropped after the office move" — observation about a state change with no entity it cleanly attaches to.
+- "There's a recurring tension between sales and ops on lead-handoff timing" — meta-observation about an organisational dynamic.
+- "Joel's writing style shifts to formal English when discussing legal matters" — a behavioural pattern that doesn't fit `:Preference`.
+
+Each `:Insight` carries `summary` (one sentence, ≤200 chars), `kind` (free-text classifier label), `confidence` (Sonnet self-rated 0–1), plus the standard provenance stamps. `:DERIVED_FROM` edges link to the SPECIFIC messages that supported the insight (not the entire conversation).
+
+```cypher
+MERGE (i:Insight {insightId: $insightId})
+  ON CREATE SET
+    i.summary         = $summary,
+    i.kind            = $kind,
+    i.confidence      = $confidence,
+    i.accountId       = $accountId,
+    i.source          = 'whatsapp',
+    i.createdByAgent  = 'whatsapp-import',
+    i.createdBySession = $sessionId,
+    i.createdAt       = datetime(),
+    i.scope           = 'admin'
+WITH i, $messageIds AS mids
+UNWIND mids AS mid
+MATCH (m:Message {messageId: mid})
+MERGE (i)-[d:DERIVED_FROM]->(m)
+  ON CREATE SET d.createdAt = datetime()
+```
+
+This Cypher runs through `mcp__memory__memory-write` (single-node + relationships payload), not through `memory-archive-write`. The agent does not author this Cypher directly — the schema-aware writer translates the structured payload.
+
+## What pass 2 does NOT do
+
+- **Does not summarise the conversation.** A `:KnowledgeDocument` summary is the wrong shape per `feedback_archives_are_not_documents.md`.
+- **Does not score sentiment** per message. Sentiment is noise at the per-message level. Conversation-level emotional tone may show up as a `:Insight` if genuinely interesting.
+- **Does not aggregate across conversations.** Each export ingests in isolation. Cross-conversation patterns are a query-time concern using the `createdBySession` provenance index.
+- **Does not invent identifiers.** A `:MENTIONS` candidate without a `memory-search` hit is dropped, never auto-minted as a placeholder Person.
+
+## Operator-side discipline
+
+The operator may interrupt at any prompt during pass 2 and ask "skip the insight pass, just commit the messages." The skill obeys — no retroactive insight scan, no "let me extract a few quick ones first." Pass 1's Conversation+Messages is independent and complete on its own. The insight pass is opt-out at any prompt.

package/payload/platform/templates/specialists/agents/database-operator.md

@@ -107,6 +107,7 @@ The classifier maps document sections to typed ontology labels. It does not inve
 Per-source archive imports keep their own skill because their CSVs already encode entity types deterministically and need no LLM classifier. Currently shipped:
 
 - **linkedin-import** — LinkedIn Basic Data Export. Ships with references for `Profile.csv` and `Connections.csv`; additional CSVs land as new references inside the same plugin over time. Path: `platform/plugins/linkedin-import/skills/linkedin-import/SKILL.md`. Load via `plugin-read` before any ingestion.
+- **whatsapp-import** — WhatsApp `_chat.txt` export ingestion. Imports historical Conversation + Messages with chronological NEXT chain via `memory-archive-write` (archiveType=`whatsapp-export`), then derives typed insights (mentions, preferences, commitments, observed relationships) inline through existing memory tools. Distinct from the live `whatsapp` plugin (Baileys QR pairing, in-memory store). Path: `platform/plugins/whatsapp-import/skills/whatsapp-import/SKILL.md`. Load via `plugin-read` before any ingestion.
 
 Future CRM-type seed plugins (HubSpot, Salesforce, Pipedrive, iCloud contacts, Gmail CSV, etc.) will ship under the same pattern — each as its own opt-in plugin, each with its own `SKILL.md` path under `platform/plugins/<name>/skills/`. When the admin adds a new archive-import skill, its PLUGIN.md will name itself here and in the admin's `<plugin-manifest>`. No prompt change required.