@rubytech/create-realagent 1.0.828 → 1.0.829

package/payload/platform/plugins/whatsapp-import/skills/whatsapp-import/SKILL.md

@@ -1,124 +0,0 @@
----
-name: whatsapp-import
-description: Single-phase WhatsApp `_chat.txt` ingest contract (Task 891). Confirms the owner + every distinct sender against existing `:AdminUser` / `:Person` nodes (no auto-creation), then invokes the deterministic Bash entry `whatsapp-ingest.sh`. The script parses the export, sessionizes at gap-hours boundaries, classifies each session via Haiku (mode='chat') into `:Section:Conversation` chunks with summary + topic keywords, and writes them under a parent `:ConversationArchive` MERGEd on `conversationIdentity = sha256(accountId + ":" + sortedParticipantElementIds)`. Re-imports are delta-append: prior chunks are never touched; only messages after `lastIngestedMessageHash` flow through the pipeline. Triggers when the operator drops a `_chat.txt` file or its containing export folder into chat. Distinct from the live `whatsapp` plugin (Baileys QR pairing).
----
-
-# WhatsApp Import — Conversation Archive
-
-Single-phase ingest. The deterministic Bash entry parses the export, splits it into sessions, classifies each session into topic-bounded `:Section:Conversation` chunks via Haiku, and writes everything under a parent `:ConversationArchive`. Insight derivation (`:Observation` / `:Task` / `:Preference` / `:MENTIONS`) is deferred to a separate follow-up task that operates on chunks rather than per-message rows.
-
-## Owner + all-participants confirmation (mandatory first step)
-
-A `_chat.txt` carries N distinct senders (1 owner + N-1 others). Every distinct senderName must resolve to an existing `:AdminUser` or `:Person` elementId before the script runs — the writer LOUD-FAILs on any unresolved sender.
-
-The flow:
-
-1. **Preview the senders.** Call `mcp__memory__whatsapp-export-preview` with the operator-supplied path (read-only, no Cypher writes):
-   ```json
-   {
-     "filePath": "/abs/path/to/_chat.txt",
-     "timezone": "Europe/London"
-   }
-   ```
-   Returns counters + the sender histogram. Surface to the operator as one chat message — counters and the histogram, no prose.
-
-2. **List candidate `:AdminUser` and existing `:Person`** rows for the senders via `mcp__graph__maxy-graph-read_neo4j_cypher`.
-
-3. **Iterate the histogram, one operator question per distinct senderName.** For each sender: `"Sender '<name>' (<count> messages) — pick existing :AdminUser/:Person or block?"`. The operator either picks an existing elementId or names "block" (refuses to map to a node). **Never auto-create a `:Person`** — the operator must confirm a canonical node, mirroring `feedback_archives_are_not_documents.md`'s closed-set discipline.
-
-4. **Identify the owner** from the resolved set — the operator who exported the chat. Echo back: `"Owner = :AdminUser <name> (<elementId>); other participants = <list>. Confirm yes/no."`
-
-5. **Persist the resolved IDs** as `--owner-element-id` + `--participant-person-ids <csv>` for the script call.
-
-DM and group follow the identical flow. A 1:1 chat resolves 2 senders; a group resolves N senders. Identity (`conversationIdentity = sha256(accountId + ":" + sortedParticipantElementIds.join(","))`) is identical regardless of group size — DM and group use the same MERGE key.
-
-## Step 2 — invoke the script
-
-Single Bash call:
-
-```bash
-bash platform/plugins/whatsapp-import/bin/whatsapp-ingest.sh <archive.zip|dir|_chat.txt> \
-  --owner-element-id <id> \
-  --participant-person-ids <id1>,<id2>,... \
-  --scope <admin|public>
-```
-
-Optional flags:
-- `--account-id <id>` — explicit account id when more than one exists under `data/accounts/` (Phase 0 has one).
-- `--timezone <iana>` — IANA zone for timestamps (default `Europe/London`).
-- `--date-format <DD/MM/YY|MM/DD/YY|DD/MM/YYYY|MM/DD/YYYY>` — override auto-detect for ambiguous locales.
-- `--session-gap-hours <N>` — gap threshold (in hours) used to split parsed messages into sessions for chunking (default `12`). Smaller values produce more sessions, more Haiku calls, finer chunks; larger values group more messages per session.
-
-The script:
-- Unzips the archive if needed; locates `_chat.txt`.
-- Parses the file deterministically (year shape, sender/body grammar, timezone offset, U+200E/U+200F bidi-strip).
-- Computes the source file's `archiveSha256` (provenance + cleanup discriminator).
-- Validates every distinct parsed senderName against the closed set of `{owner, participants...}` candidate names. Any miss LOUD-FAILs `parser-miss reason="senderName=<verbatim> not in confirmed participant set ..."`.
-- Computes `conversationIdentity` from accountId + sorted participant elementIds.
-- Looks up any prior `:ConversationArchive` carrying that identity → reads `lastIngestedMessageHash`. If found, slices parsed lines after the cursor (delta-append). Cursor not found → `FAIL delta-cursor-missing`. Cursor at last line → empty-delta noop (exit 0, no writes).
-- Sessionizes the delta lines at the operator-supplied gap-hours boundary.
-- For each session: renders as turn-attributed text (`[ts] Sender: body\n…`) and calls `memory-classify` with `mode='chat'`. Returns one or more `:Section:Conversation` chunk specs with `summary`, `keywords[]`, `firstMessageAt`, `lastMessageAt`, `participantNames[]`, `messageCount`, `body` (verbatim turn-attributed text).
-- Calls `memory-ingest` with `parentLabel='ConversationArchive'`. Server MERGEs the parent on `conversationIdentity`, MERGEs `:PARTICIPANT_IN` edges from each confirmed participant, drops any prior chunks stamped with this `archiveSha256` (idempotency for re-running the same export bytes), CREATEs new chunks, extends the existing `:NEXT` chain from its tail, advances `lastIngestedMessageHash` + `lastIngestedMessageAt`.
-
-NO insight pass runs. Phase 2 (operator-driven `:Observation` / `:Task` / `:Preference` derivation against chunks) is its own follow-up task with its own skill.
-
-## Three operator messages per ingest
-
-After the script succeeds, formulate the three operator-facing messages from the JSON summary on stdout (one operator message per surfaceable phase):
-
-1. **Parse summary.** `Parsed <archiveSourceFile>: <parsed> messages across <sessions> sessions, date range <dateRange.first> → <dateRange.last>. Participants: <senderHistogram[i].name (count), …>.`
-2. **Classify summary.** `Classified into <chunks> chunks, covering: <topicKeywords[0], topicKeywords[1], …>.`
-3. **Write summary.** `Created :ConversationArchive <archiveElementId> with <chunks> :Section:Conversation chunks (NEXT chain length <chunks - 1>). Participants linked via :PARTICIPANT_IN: <participantsLinked>.`
-
-For an empty-delta re-import (`delta.kind === "empty-delta"`): emit only message 1 + a noop line `noop reason="no new messages since <priorLastIngestedMessageAt>"`.
-
-## Stdout JSON shape (success)
-
-```json
-{
-  "archiveElementId": "4:abcd…:42",
-  "conversationIdentity": "<sha256-hex>",
-  "archiveSha256": "<sha256-hex>",
-  "archiveSourceFile": "_chat.txt",
-  "parsed": 1707,
-  "mediaSkipped": 0,
-  "systemSkipped": 0,
-  "delta": { "kind": "first-ingest|delta|empty-delta", "deltaStart": 0, "deltaMessages": 1707 },
-  "sessions": 38,
-  "chunks": 142,
-  "nextEdgesCreated": 141,
-  "participantsLinked": 2,
-  "dateRange": { "first": "2024-01-15T09:30:00+00:00", "last": "2026-04-30T18:42:00+01:00" },
-  "senderHistogram": [{ "name": "Joel", "count": 812 }, { "name": "Adam", "count": 895 }],
-  "topicKeywords": ["pricing", "scheduling", "introductions", "..."],
-  "ms": 6800
-}
-```
-
-## Failure path — single FAIL line
-
-- **Exit non-zero** + one stderr line: `[whatsapp-import] FAIL phase=<argv|parse|classify|delta-cursor-missing|memory-ingest|uncaught> reason="..."`. Surface this verbatim to the operator and yield. **Do not retry.** The archive-ingest-surface-gate denies parser-source edits, JS test runners, and the legacy `whatsapp-export-parse` / `whatsapp-export-insight-write` MCP tools — none of those are escape hatches in your surface.
-- `parser-miss` LOUD-FAIL: an unconfirmed senderName slipped through. Either re-run with the missing :Person elementId added to `--participant-person-ids`, or report a parser bug.
-- `delta-cursor-missing` LOUD-FAIL: the prior `lastIngestedMessageHash` is not present in the re-export. Either the operator deleted prior messages from the archive, or this is a different chat. Investigation required — never re-run blindly.
-
-## Idempotency
-
-- **Re-running the same export bytes** is a no-op: the cleanup-by-`archiveSha256` step drops THIS export's prior chunks and re-creates them with identical content. `:NEXT` chain length unchanged. Counters: `chunks` non-zero, `delta.kind` either `first-ingest` (if no prior archive) or `delta` (cursor advanced past previous run).
-- **Re-running with appended messages** (a fresh export from the same chat with new messages at the tail): cursor lookup finds the prior `lastIngestedMessageHash`, slices new messages, sessionizes only those, and appends new chunks at the tail of the existing `:NEXT` chain. Pre-existing chunks are never touched (their `archiveSha256` differs from this run's).
-
-## Verification (post-write)
-
-Run via `mcp__graph__maxy-graph-read_neo4j_cypher`:
-
-- `MATCH (a:ConversationArchive { conversationIdentity: $cid }) RETURN elementId(a), a.lastIngestedMessageAt, a.lastIngestedMessageHash` — agrees with the JSON summary.
-- `MATCH (a:ConversationArchive { conversationIdentity: $cid })-[:HAS_SECTION]->(c:Section:Conversation) RETURN count(c)` — equals `chunks`.
-- `MATCH p=(:Section:Conversation)-[:NEXT*]->(:Section:Conversation) WHERE startNode(relationships(p)[0]).archiveSourceFile = $file WITH max(length(p)) AS chain RETURN chain` — equals `chunks - 1` for a fresh-only ingest, or longer for a delta-append (full chain since first ever ingest).
-- `MATCH (p)-[:PARTICIPANT_IN]->(:ConversationArchive { conversationIdentity: $cid }) RETURN count(p)` — equals `participantsLinked` after a first-ingest, or the running total of all confirmed participants ever.
-- Phase 1 wrote ZERO observations: `MATCH (o:Observation)-[:OBSERVED_IN]->(:ConversationArchive { conversationIdentity: $cid }) RETURN count(o)` — should be 0 today (Phase 2 deferred).
-
-## What this is not
-
-- **Not** the live `whatsapp` plugin. That plugin (Baileys QR pairing) holds messages in an in-memory store cleared on restart. This plugin imports historical exports into Neo4j as persistent graph nodes.
-- **Not** a media-transcription pipeline. Voice notes, photos, PDFs are skipped at parse with a counter logged.
-- **Not** an insight-extraction pass. Phase 2 (`:Observation` / `:Task` / `:Preference` / `:MENTIONS` derivation, anchored to chunks) ships in its own task.
-- **Not** automatic. The owner + all-participants confirmation gate is mandatory before any line is written, mirroring `feedback_archives_are_not_documents.md`'s closed-set discipline.

package/payload/platform/plugins/whatsapp-import/skills/whatsapp-import/references/conversation-archive-shape.md

@@ -1,143 +0,0 @@
-# Conversation Archive — graph shape, identity, edges, delta protocol
-
-The reference document for the `:ConversationArchive` shape introduced by Task 891. This is the source of truth for what the graph looks like after a `whatsapp-ingest.sh` run — the SKILL.md prescribes the workflow; this file specifies the schema.
-
-## Labels
-
-| Label | Role | MERGE key | Schema constraint |
-|---|---|---|---|
-| `:ConversationArchive` | Parent node — one per chat | `conversationIdentity` | `FOR (a:ConversationArchive) REQUIRE a.conversationIdentity IS UNIQUE` |
-| `:Section:Conversation` | Topic-bounded chunk of messages | (no MERGE — CREATE only) | inherits `:Section` indices |
-
-`:Section:Conversation` is the chat-mode counterpart of document-mode `:Section:Chapter` etc. Same `:Section` base label, same indexing, different secondary label.
-
-## Identity formula
-
-```
-conversationIdentity = sha256(accountId + ":" + sortedParticipantElementIds.join(","))
-```
-
-- Stable across re-exports — same accountId + same operator-confirmed participant set always produces the same identity, regardless of the source file's bytes.
-- DM and group are identical — the difference is array length.
-- Participant order is sorted before joining, so the operator can supply elementIds in any order.
-
-## Required indices
-
-```
-INDEX :ConversationArchive(accountId)
-INDEX :ConversationArchive(createdBySession)
-```
-
-Plus the constraint above (which doubles as a uniqueness index on `conversationIdentity`).
-
-## Properties on `:ConversationArchive`
-
-| Property | Type | Source | When set |
-|---|---|---|---|
-| `conversationIdentity` | string (sha256-hex) | derived | ON CREATE |
-| `accountId` | string (UUID) | argv | ON CREATE |
-| `scope` | string (`admin` / `public`) | argv | ON CREATE |
-| `summary` | string | classifier (synthetic) | ON CREATE |
-| `keywords` | string[] | classifier (aggregated across sessions) | ON CREATE |
-| `embedding` | float[] | embed(summary) | ON CREATE |
-| `archiveSourceFile` | string (basename) | this export | ON CREATE |
-| `createdAt` | ISO 8601 | this run | ON CREATE |
-| `createdByAgent` | string | constant `"whatsapp-import"` | ON CREATE |
-| `createdBySession` | string (UUID) | env / argv | ON CREATE |
-| `source` | string | constant `"whatsapp"` | ON CREATE |
-| `updatedAt` | ISO 8601 | this run | ON CREATE / ON MATCH |
-| `lastIngestedMessageHash` | string (sha256-hex) | derived from last delta line | ON CREATE / ON MATCH |
-| `lastIngestedMessageAt` | ISO 8601 | last delta line's `dateSent` | ON CREATE / ON MATCH |
-| `lastIngestedBySession` | string (UUID) | this run | ON CREATE / ON MATCH |
-| `lastIngestedArchiveSha256` | string (sha256-hex) | this export's file bytes | ON CREATE / ON MATCH |
-
-## Properties on `:Section:Conversation` (each chunk)
-
-| Property | Type | Source |
-|---|---|---|
-| `accountId` | string | inherited |
-| `title` | string | classifier |
-| `body` | string | classifier (verbatim turn-attributed text — `[ts] Sender: body\n…`) |
-| `bodyPreview` | string (≤150 chars) | first 150 chars of body |
-| `position` | int | chunk index within this run |
-| `scope` | string | inherited |
-| `embedding` | float[] | embed(body) |
-| `summary` | string | classifier (1–3 sentences) |
-| `keywords` | string[] | classifier |
-| `firstMessageAt` | ISO 8601 | first `[ts]` in chunk |
-| `lastMessageAt` | ISO 8601 | last `[ts]` in chunk |
-| `participantNames` | string[] | distinct senderNames in chunk |
-| `messageCount` | int | message count in chunk |
-| `archiveSha256` | string (sha256-hex) | this export's file bytes (cleanup discriminator) |
-| `archiveSourceFile` | string (basename) | this export |
-| `createdAt` | ISO 8601 | this run |
-| `createdByAgent` / `createdBySession` / `source` | provenance | this run |
-
-## Edges
-
-| Edge | From | To | Cardinality | When written |
-|---|---|---|---|---|
-| `:HAS_SECTION` | `:ConversationArchive` | `:Section:Conversation` | one per chunk | every run |
-| `:NEXT` | `:Section:Conversation` | `:Section:Conversation` | chunks − 1 (chronological chain across all sessions across all runs) | extending tail per run |
-| `:PARTICIPANT_IN` | `:Person` / `:AdminUser` | `:ConversationArchive` | one per confirmed participant | MERGEd every run (idempotent) |
-
-Every edge carries `createdAt`, `createdByAgent`, `createdBySession`, `source`, plus `archiveSha256` for HAS_SECTION + NEXT (so cleanup-by-archiveSha256 catches the right edges).
-
-## Delta-append protocol
-
-```
-                        first ingest (or empty graph)
-                                │
-                                ▼
-   parsed_lines  ──── all of them ────► sessionize ──► classify ──► memory-ingest
-                                                                       │
-                                                                       ▼
-                                                       :ConversationArchive (NEW)
-                                                       └── :HAS_SECTION ──► chunks (NEXT chain length K-1)
-
-
-                        re-import (delta)
-                                │
-                                ▼
-   parsed_lines  ──┐
-                   │
-                   ▼
-              find cursor where deriveMessageContentHash(line) == archive.lastIngestedMessageHash
-                   │
-        ┌──────────┼──────────┬───────────────────┐
-        │          │          │                   │
-       found     missing    empty (cursor at last line)
-        │          │          │
-        ▼          ▼          ▼
-   slice from   FAIL    noop (exit 0,
-   cursor+1     non-zero  no writes)
-        │
-        ▼
-   delta_lines ──► sessionize ──► classify ──► memory-ingest (parentLabel='ConversationArchive')
-                                                  │
-                                                  ▼
-                                  :ConversationArchive (MERGE on conversationIdentity)
-                                  └── :HAS_SECTION ──► NEW chunks
-                                  ── :NEXT extends from prior tail
-                                  prior chunks unchanged
-                                  cursor advances
-```
-
-## Why the cursor is content-only (not file-byte-based)
-
-`lastIngestedMessageHash = sha256(dateSent + "|" + NFKC-trim-lower(senderName) + "|" + body)`. The hash deliberately excludes the source file's bytes — a fresh re-export of the same chat has different file bytes (different SHA-256 of the file) but the SAME message tuples. Without content-only hashing, every delta-import would `delta-cursor-missing` because the file SHA-256 always changes.
-
-## Why DM and group are identical
-
-The brief's "DM-only" Task 887 §A0 contract was a workaround for the per-message writer's auto-Person leak. Under the chunked archive shape, the writer never auto-creates anyone — every participant is operator-confirmed up front. The 2-vs-3-vs-N participant case is just the array length on the right of the identity formula. No special-casing.
-
-## Provenance discipline (for cleanup correctness)
-
-Every node and edge written by this pipeline is stamped with:
-- `source = 'whatsapp'`
-- `createdByAgent = 'whatsapp-import'`
-- `createdBySession = <this run's session UUID>`
-- `archiveSha256 = <this export's file SHA-256>` (chunks + HAS_SECTION + NEXT only — the parent records the LAST archiveSha256 separately)
-- `archiveSourceFile = <this export's basename>` (parent + chunks)
-
-The cleanup-by-`archiveSha256` step in memory-ingest's chat path drops only chunks whose `archiveSha256` matches THIS run's. Re-running the same export bytes is a no-op idempotently; re-running with a fresh delta export (different bytes, different SHA-256) leaves prior chunks untouched and appends new ones at the tail.

package/payload/platform/plugins/whatsapp-import/skills/whatsapp-import/references/export-parse.md

@@ -1,109 +0,0 @@
-# Reference: `_chat.txt` parsing — implementation reference
-
-> **This is no longer operator instruction.** The agent does NOT walk this grammar in its own LLM turn. Parsing runs deterministically in [`platform/plugins/whatsapp-import/lib/src/parse-export.ts`](../../../lib/src/parse-export.ts), invoked in-process by [`bin/ingest.mjs`](../../../bin/ingest.mjs) (which the operator calls via [`bin/whatsapp-ingest.sh`](../../../bin/whatsapp-ingest.sh) — the single deterministic Bash entry). The legacy MCP wrapper is blocked at the harness gate. The vitest grid in [`lib/src/__tests__/parse-export.test.ts`](../../../lib/src/__tests__/parse-export.test.ts) is the executable contract; this prose is the human-readable companion. Extend the grammar by adding a failing test first.
-
-WhatsApp's "Export Chat" produces a UTF-8 text file with a deterministic line grammar. This reference describes what the parser library does when it converts that file into the `{senderName, dateSent, body, sequenceIndex}[]` structure the SKILL.md consumes.
-
-## File-open invariants
-
-1. **UTF-8 only.** Open the file with explicit UTF-8 decoding. On encoding error, abort the import with a named LOUD-FAIL — never silently substitute or corrupt bodies. WhatsApp's modern apps emit UTF-8 reliably; an encoding error usually means the operator manually edited the file with a tool that broke it. Surface the named error so they can re-export.
-2. **No size cap from the parser.** The parser handles arbitrarily large files; the [SKILL.md](../SKILL.md)'s 100-message selective-ingest gate is the operator-facing compression layer.
-3. **Compute `archiveSourceFile = sha256(file bytes)` first.** The hash drives `conversationId` and lets re-imports of the same archive land idempotently.
-
-## Line grammar
-
-Every message line begins with a square-bracketed timestamp prefix followed by `<Sender>: <body>`:
-
-```
-[DD/MM/YYYY, HH:MM:SS] <Sender>: <body>      ← modern WhatsApp default (4-digit year)
-[DD/MM/YY, HH:MM:SS] <Sender>: <body>        ← legacy exports (2-digit year)
-```
-
-- **Day/month ordering.** `DD/MM` is the WhatsApp default everywhere except US iOS, which emits `MM/DD`. The parser auto-detects from the first prefix-matching line when `dateFormat` is omitted: probe DD/MM first; if range-valid, lock DD/MM; otherwise lock MM/DD. The lock is per-file — a single export never mixes orderings (the locale is set by the device that generated the export). Manually concatenated multi-locale archives are an explicit out-of-scope: pass `dateFormat` to override.
-- **Year shape.** Both 2-digit (`\d{2}` → `2000+yy`) and 4-digit (`\d{4}` → as-is) years are accepted by the same regex (`\d{2,4}`). A single file may hold both shapes; year semantics are resolved per-line from the captured length, not per-file.
-- **Time.** `HH:MM:SS` 24-hour; older exports may emit `HH:MM` (no seconds — treat as `:00`).
-- **Sender.** Saved contact name, phone number with country code (`+44 7700 900123`), or `You` for legacy operator-sent messages.
-- **Body.** Message text, possibly multi-line.
-
-Trim trailing whitespace from each line before parsing.
-
-## Multi-line bodies
-
-A body that wraps to multiple lines continues onto subsequent lines that do **not** match the timestamp-prefix pattern. The parser must accumulate these continuation lines into the previous message's body, joining with `\n`. End-of-message is detected by the next timestamp prefix or end-of-file.
-
-```
-[14/03/26, 10:15:23] Joel: Quick question about the deck —
-do you have the v3 PDF anywhere?
-I checked Drive and only see v2.
-[14/03/26, 10:16:01] Sarah: Sec, will dig it out
-```
-
-Yields two messages:
-- `Joel: "Quick question about the deck —\ndo you have the v3 PDF anywhere?\nI checked Drive and only see v2."`
-- `Sarah: "Sec, will dig it out"`
-
-## System messages — skip with counter
-
-WhatsApp injects system-generated lines into the export for group events, contact changes, and security messages. These lines match the timestamp prefix but are **not** sent by a person and have no first-class graph value. Skip them at parse time, increment `systemSkipped`, and do not pass to `memory-archive-write`.
-
-Patterns to recognise (English; localisation expands the list):
-
-- `<Sender> created group "<name>"`
-- `<Sender> changed the subject from "<old>" to "<new>"`
-- `<Sender> changed this group's icon`
-- `<Sender> added <other>`
-- `<Sender> removed <other>`
-- `<Sender> left`
-- `<Sender>'s security code changed.`
-- `Messages and calls are end-to-end encrypted. ...` (the conversation header)
-- `You deleted this message.`
-- `This message was deleted.`
-
-Heuristic for the parser: if the body contains no spaces between `<Sender>` and a verb-phrase token, AND the body lacks a colon-after-sender separator, treat as a system message. Conservative — when uncertain, prefer to ingest as a message; the insight pass tolerates noisy bodies better than the parser would tolerate dropped real messages.
-
-## Media attachments — skip with counter
-
-Lines whose body indicates a media-only message (no text content) get skipped at parse time, increment `mediaSkipped`. Patterns:
-
-- `<Media omitted>` (when the operator chose "Without Media" on export)
-- `IMG-<digits>-<digits>.jpg (file attached)` / `.jpeg` / `.png` / `.heic` / `.gif`
-- `VID-<digits>-<digits>.mp4 (file attached)`
-- `PTT-<digits>-<digits>.opus (file attached)` (voice notes)
-- `AUD-<digits>-<digits>.opus (file attached)` (audio)
-- `STK-<digits>-<digits>.webp (file attached)` (stickers)
-- `<filename>.pdf (file attached)`
-- `<filename>.docx (file attached)`
-- `‎<...> attached: <filename>` (alternative format on some platforms)
-
-Mixed messages (text + media reference in one body) are kept as messages — only pure-media-only lines are skipped. The text body is retained.
-
-## Forwarded messages
-
-A forwarded message is prefixed with the invisible Unicode `‎` (U+200E LEFT-TO-RIGHT MARK) followed by metadata WhatsApp injects. Parse the body normally; the LRM character is preserved in `body` (the insight pass's classifier sees it as benign). Do not strip — the raw body's fidelity matters for downstream queries.
-
-## Edge cases
-
-- **Empty body** (timestamp prefix followed by sender colon but no text). Rare. Skip with `systemSkipped` increment — usually corresponds to a deleted message stub.
-- **Leading BOM** (U+FEFF at file start). Strip before parsing the first line.
-- **Mixed line endings** (`\r\n` vs `\n`). Normalise to `\n` before tokenisation.
-- **Sender containing a colon** (e.g., a contact named "Joel: Work"). The grammar splits on the FIRST `: ` (colon-space) after the timestamp prefix's closing `]`. Subsequent colons in the sender or body are preserved verbatim.
-
-## Parser output shape
-
-The parser returns `{conversationId, archiveSourceFile, parsedLines[], counters}` where:
-
-- `parsedLines[]: Array<{senderName: string, dateSent: string (ISO 8601 with operator-supplied timezone), body: string, sequenceIndex: number}>`
-- `counters: {parsed: n, systemSkipped: n, mediaSkipped: n, parseErrors: n}`
-
-The skill consumes this directly. The `messageId` is computed by the skill (not the parser) so the `lineHash` covers the original raw line, not the post-parse normalised body.
-
-## When to LOUD-FAIL
-
-The parser throws (and `whatsapp-export-parse` returns `isError: true`) on:
-
-- Encoding error at file open (UTF-8 decode fails — the parser uses `TextDecoder` with `fatal: true`, so any invalid byte sequence aborts loudly rather than silently substituting U+FFFD).
-- Empty file or zero parsed lines after walking the file (the file isn't a `_chat.txt`). The thrown error and the `[whatsapp-import] parse-grammar-miss first-line="<sample>"` stderr line both carry a sanitised first-line sample (control chars stripped, truncated to 80 chars) so the operator can recognise the offending header shape without re-running with a debugger.
-- A timestamp prefix matches but the body parse fails (no `: ` separator after the closing `]` AND no system-pattern match) — emits `parse-error file=<...> line=<n> reason=no-sender-body-separator content="<...>"`.
-- Missing required input (`accountId`, `timezone`).
-
-Never silently drop data the parser couldn't classify. The operator chooses to skip; the parser does not choose for them.