overleaf-review 0.3.0 → 0.5.0

package/README.md

@@ -35,14 +35,24 @@ workflow and carries the review layer Git can't represent.
 - 📥 **`pull`** — read comments + tracked changes (with anchors) into a Git-friendly sidecar
   (`.overleaf/reviews.md` + `.json`), so your tools have every co-author note in context.
 - 📤 **`push`** — turn local edits into **tracked-change suggestions**, mapping files to Overleaf
-  docs by path (one file, or every changed `.tex` at once). `--dry-run` previews the exact ops;
-  `--direct` sends plain edits instead of suggestions.
+  docs by path (one file, or every changed `.tex` at once). A saved Base/Local/Live merge keeps
+  non-overlapping co-author edits, while overlapping edits and active tracked-change ranges stop
+  the push for inspection. `--direct` sends plain edits instead of suggestions.
 - 🔄 **`fetch`** — write Overleaf's current text back down into your repo (read-only on Overleaf, so
-  it cannot disturb a single comment or tracked change).
+  it cannot disturb a single comment or tracked change) and save the synchronization base used by
+  future pushes. Existing local files are snapshotted before replacement.
+- 📋 **`review plan` / `review submit`** — save a complete, binding push plan, inspect it, then
+  submit it only if the local file, saved base, live document version, and review ranges are still
+  unchanged.
 - 🖼️ **`upload`** — push figures, PDFs, or new files into Overleaf.
 - 💬 **`comment` / `reply` / `resolve` / `delete-comment` / `delete-message`** — full comment
   control: start a thread, reply, resolve/reopen, delete a whole thread or a single message.
-- ✅ **`accept` / `reject`** — act on your collaborators' tracked changes from the CLI.
+  Identical recent comments and replies are retry-safe by default; `--force` deliberately posts
+  another copy.
+- ✅ **`accept` / `reject`** — act on collaborators' tracked changes and verify removal of every
+  requested change id before reporting success; rejection also verifies the exact resulting text.
+- 🧾 **Review receipts** — pushes, accept/reject actions, comments, and replies leave durable JSON
+  records under `.overleaf/receipts/` showing what was verified or left ambiguous.
 - 🔑 **`login`** — validated auth stored outside your repo (chmod 600); `--browser` mode is
   institutional-SSO friendly.
 
@@ -60,9 +70,11 @@ ops that Overleaf *transforms the review ranges against*, so comments and tracke
 
 | Need | Command |
 | --- | --- |
-| Overleaf text → your repo | `fetch` |
+| Begin a local review session | `review start` |
+| Overleaf text → your repo and save Base | `fetch` |
 | Your edits → Overleaf, as suggestions | `push` |
 | Your edits → Overleaf, directly | `push --direct` |
+| Inspect, then submit the exact same change | `review plan --out plan.json`, then `review submit --plan plan.json` |
 | Figures / new files → Overleaf | `upload` |
 
 Your Git repo stays a completely normal repo — commit whatever you like, `.tex` included — and
@@ -85,14 +97,28 @@ overleaf-review login --browser   # opens your Chrome, log in normally (SSO work
 # 2. Link this repo to an Overleaf project (id from the project URL)
 overleaf-review link --project 6a4c…bec5a
 
-# 3. Sync the review layer
-overleaf-review pull                        # comments + changes → .overleaf/
-overleaf-review push --dry-run              # preview local edits as suggestions
-overleaf-review push                        # send them as tracked changes
+# 3. Start from Overleaf's current text and review state
+overleaf-review review start                # fetch text + save Base + pull reviews
+
+# 4. Edit locally, then inspect and submit a binding plan
+overleaf-review review plan --out .overleaf/push-plan.json
+overleaf-review review submit --plan .overleaf/push-plan.json
+
+# Or use the safe one-shot form when a separate approval step is unnecessary
+overleaf-review push --dry-run              # transient preview; sends nothing
+overleaf-review push                        # plan, validate, send, and verify
+
+# 5. Work with review threads
 overleaf-review comment --anchor "Introduction" --message "Expand this section"
 overleaf-review resolve --thread <id>       # thread ids come from `pull`
 ```
 
+`fetch` and `review start` write Overleaf's text into the local files, so commit or otherwise
+preserve local-only work before running them. They save `.overleaf/base.json`; edit only after that
+base is established. Existing installations should run one of these commands once before their
+first safe push. If an upgraded repository already has unpushed edits, copy or stash them, run
+`fetch`, then restore the edits so the new Base remains the Overleaf version they were based on.
+
 ## 🧭 Commands
 
 | Command | What it does |
@@ -100,11 +126,16 @@ overleaf-review resolve --thread <id>       # thread ids come from `pull`
 | `login [--cookie <v>] [--browser]` | Authenticate and store your session (SSO-friendly `--browser`) |
 | `link --project <id>` | Link this repo to an Overleaf project (`.overleaf/config.json`) |
 | `pull [--out <dir>]` | Read comments + tracked changes into a sidecar |
-| `fetch [--file <f>] [--dry-run]` | Write Overleaf's text down into local files (read-only on Overleaf) |
+| `fetch [--file <f>] [--dry-run]` | Write Overleaf text locally, snapshot replaced files, and save Base (read-only on Overleaf) |
+| `review start [--file <f>] [--out <dir>]` | Fetch text, save the synchronization base, and pull the review sidecar |
+| `review plan --out <plan.json> [--file <f>] [--edits <blocks.json>] [--doc <name>] [--direct] [--allow-overlap] [--unsafe-no-base]` | Create a complete binding plan without changing Overleaf |
+| `review submit --plan <plan.json> [--acknowledge-ambiguous]` | Submit that plan only if all recorded preconditions still match |
+| `review consolidate --doc <path> --author <user-id> --out <preview.json> [--change <id> …]` | Preview consolidation of existing suggestions and save a full backup; no Overleaf changes |
+| `review consolidate --apply --plan <plan.json>` | Apply and verify a checked consolidation plan for your own suggestions |
 | `upload <path…> [--folder <name>]` | Upload figures / new files into Overleaf |
-| `push [--file <f>] [--doc <name>] [--direct] [--dry-run]` | Send local edits as tracked suggestions (all changed `.tex` if no `--file`); `--direct` for plain edits |
-| `comment --anchor <text> --message <text> [--doc <name>] [--nth <n>]` | Add a comment anchored on the given text |
-| `reply --thread <id> --message <text>` | Reply to an existing comment thread |
+| `push [--file <f>] [--doc <name>] [--direct] [--dry-run] [--plan-out <path>] [--plan <path>] [--allow-overlap] [--unsafe-no-base] [--acknowledge-ambiguous]` | Safely merge and send local edits (all changed `.tex` if no `--file`) |
+| `comment --anchor <text> --message <text> [--doc <name>] [--nth <n>] [--force]` | Add an anchored comment; recent identical retries are skipped unless forced |
+| `reply --thread <id> --message <text> [--force]` | Reply to a thread; recent identical retries are skipped unless forced |
 | `resolve --thread <id> [--reopen]` | Resolve (or reopen) a comment thread |
 | `delete-comment --thread <id>` | Delete a whole comment thread |
 | `delete-message --message-id <id> [--thread <id>]` | Delete a single message within a thread |
@@ -113,6 +144,154 @@ overleaf-review resolve --thread <id>       # thread ids come from `pull`
 
 Thread and change ids are listed by `pull` (in `.overleaf/reviews.md`).
 
+### Safe pushes
+
+A normal push compares three versions of each document:
+
+- **Base** — the Overleaf text saved by the last `fetch` or successful push;
+- **Local** — your current file;
+- **Live** — the document as it exists in Overleaf when planning.
+
+Your Base→Local intent is rebased onto Live. Non-overlapping live edits are preserved; genuinely
+overlapping Base→Local and Base→Live edits abort instead of silently undoing a collaborator's
+work. A proposed edit that intersects an active tracked-change range is also blocked by default and
+the relevant change ids are listed.
+
+For intentional sentence or paragraph rewrites, agents should preserve their chosen edit boundaries
+with `--edits`, rather than relying on automatic diff grouping. Edit the local file normally, then
+describe every change in a JSON manifest:
+
+```json
+{
+  "replacements": [
+    {
+      "before": "This is the old paragraph. Its explanation is unclear.",
+      "after": "This is the revised paragraph. Its explanation is clearer."
+    }
+  ]
+}
+```
+
+```sh
+overleaf-review review plan --file main.tex --edits blocks.json --out plan.json
+overleaf-review review submit --plan plan.json
+```
+
+Each block becomes one deletion and/or insertion, even across multiple sentences or LaTeX markup.
+Overleaf controls the resulting review UI; a replacement is generally two tracked ranges, not a
+guaranteed single accept/reject button. Keep typo corrections small, group a coherent rewrite,
+and keep independent decisions separate. Author attribution remains unchanged.
+
+`before` must match the saved Base exactly. Repeated text requires a 1-based `occurrence`.
+Blocks must be separated and must explain the entire local diff. The plan embeds the resolved
+blocks; it never rereads or silently reinterprets the manifest during submission. Concurrent edits
+outside blocks are preserved; any comment, pending suggestion, or concurrent edit touching a block
+stops planning rather than silently fragmenting it. Narrow the block or resolve the conflict.
+`--edits` requires `--file` and a saved base; it cannot be combined with `--direct` or `--unsafe-no-base`.
+
+Without explicit blocks, tracked suggestions group nearby word edits into phrase replacements. For example, rewriting
+“old model predicts low” as “revised model explains high” produces one deletion and one insertion,
+instead of six separate word operations. Isolated corrections stay small. Grouping bridges at most
+three unchanged words (40 characters), limits combined spans to 320 characters, and stops at detected
+sentence/clause boundaries, paragraph breaks, and LaTeX markup. This is a deterministic readability
+heuristic; it does not infer which scientific claims should be accepted together. An existing
+multiword insertion remains a single insertion.
+
+Unchanged comment anchors, tracked ranges, and edits made on Live since Base prevent grouping across
+them. Overlap checks use the full grouped footprint, and submission reproduces the same grouping.
+`--direct` retains the narrower word-level operations. Plans from before this grouping change must
+be regenerated; submitting an old plan does not silently regroup its approved operations.
+
+Planning and submission check a conservative range budget: existing tracked ranges plus proposed
+insert/delete operations must not exceed 2,000 per document. The preview shows both counts.
+Overlap transformations can alter the actual count, so this is a preflight guard rather than an
+exact prediction. Splitting a revision into batches does not remove accumulated pending ranges.
+Grouping applies to new pushes; it does not consolidate or accept suggestions already on Overleaf.
+
+### Consolidation of existing suggestions
+
+`review consolidate --doc main.tex --author <user-id> --out .overleaf/consolidation.json`
+captures the live text, all document ranges, and project comment threads. It reconstructs the text
+with that author's selected suggestions rejected, then plans a grouped revision back to the current
+proposed text. By default it considers all current suggestions by that author, including older
+fragments; repeat `--change` to select a subset. Use the author user ID from range metadata.
+
+The artifact records both text hashes, a reverse/forward reconstruction proof, projected range
+counts, and any comment or unselected-suggestion blockers. It preserves the *current pending
+proposal*, not a historical review state before older suggestions were absorbed. Consolidation
+would assign new IDs and timestamps. Counts remain projections until server transformations are
+verified. Touching comment anchors or unselected changes blocks the candidate; these are not
+automatically removed or accepted.
+
+For offline analysis, use `--snapshot <file.json>` instead of `--doc`. The snapshot must contain
+`projectId`, `docId`, `docPath`, `version`, `text`, `ranges: { changes, comments }`, and `threads`;
+the preview embeds this complete object as `backup`. A normal review sidecar alone is insufficient.
+
+Planning sends nothing. Inspect the complete plan, then apply it separately:
+
+```sh
+overleaf-review review consolidate --apply --plan .overleaf/consolidation.json
+```
+
+Only suggestions owned by the authenticated account can be reapplied. Blocked/no-reduction plans
+cannot be submitted. The command reproduces the plan from its backup and requires unchanged text,
+document version, ranges, and project threads. It journals the full snapshot before sending undo
+and reapply operations together in one update. It never accepts suggestions as a prerequisite.
+
+Verification checks the proposed text, text with selected suggestions rejected, text with all
+suggestions rejected, disappearance of old IDs, new-ID attribution, reduced range count, and
+unchanged unselected ranges, comment anchors and thread messages. New IDs and timestamps are
+expected. Any uncertain send or failed verification quarantines subsequent attempts: inspect the
+receipt and live document before manually reconciling it; there is no automatic rollback/retry.
+Reusing an already successful plan reports its receipt without sending it again.
+
+Coordinate a quiet editing window for consolidation. Overleaf has no conditional transaction
+covering text, ranges and threads, so preflight cannot prevent every last-moment collaborator race;
+readback detects discrepancies but cannot make an already-sent operation un-happen. A backup is
+recovery evidence, not a promise of automatic review-history restoration.
+
+Development check `npm run probe:consolidation-model` exercises pinned upstream range logic in
+memory without accessing an Overleaf project. The opt-in `npm run probe:review-live -- --project
+<id> --confirm-test-project` creates and retains a new scratch document in the specified test
+project, verifies consolidation and protected review state, and checks that existing documents
+were unchanged. It does not require or spoof another account. Foreign-author preservation is also
+covered by the upstream model check. Never run this probe against a manuscript project.
+
+### Push overrides and submission
+
+Use `--allow-overlap` only after inspecting those tracked changes. `--unsafe-no-base` is a deliberate
+legacy escape hatch that treats the current Live document as Base; it loses the protection against
+co-author edits made since your local file was obtained. `--direct` changes how the validated ops
+are represented in Overleaf, not the merge and conflict checks.
+
+For a separate approval step, `review plan --out plan.json` stores the full operation list and its
+preconditions, active tracked ranges, and any comment overlaps. `review submit --plan plan.json`
+re-derives the plan from Base/Local/Live and aborts if preflight finds that the project, local source,
+synchronization base, live document version/content, or active review ranges changed. Plans contain
+manuscript fragments, so store or commit them with the same care as the source itself. `push
+--plan-out <path>` and `push --plan <path>` expose the same workflow through the one-level command.
+
+Overleaf can transform an update that races with a collaborator after the final preflight. The tool
+therefore reads every result back and fails rather than claiming success when the exact planned text
+is not present. A multi-file submission is not transactional; if a later file fails, its receipt
+identifies every earlier file already verified on Overleaf.
+
+Pushes, accept/reject actions, comments, and replies write audit receipts beneath
+`.overleaf/receipts/`. Keep an ambiguous receipt: it means the request may have reached Overleaf but
+readback could not prove the result, so inspect the live project before retrying. For comments and
+replies, an identical recent message is treated as a retry and skipped; pass `--force` only when a
+second identical message is intentional.
+
+An ambiguous push quarantines its affected documents even if a newly generated plan has a different
+timestamp or tracked-change seed. After waiting for delayed updates and manually reconciling the
+receipt against live Overleaf, `--acknowledge-ambiguous` explicitly clears that submission-time
+guard; it should never be used as an automatic retry flag.
+
+Mutating commands are serialized by a working-tree-specific lock in the system temporary directory,
+preventing two local agents from racing through the same preflight. A normal exit removes the lock.
+If a process crashes and leaves it behind, the next command prints its exact path; inspect any
+relevant receipt and the live Overleaf document before removing it manually.
+
 ## 🧠 How it works
 
 overleaf.com's editor speaks an old **socket.io 0.9** protocol over a WebSocket. The client
@@ -122,8 +301,12 @@ overleaf.com's editor speaks an old **socket.io 0.9** protocol over a WebSocket.
 - a **tracked change** is an insert/delete op with a `meta.tc` flag;
 - a **comment** is a `c` op plus a REST post of the message text.
 
-`push` diffs your local file against Overleaf's current content and translates the hunks into
-sequential OT ops, so your edits land as reviewable suggestions rather than silent changes.
+`push` performs the Base/Local/Live merge described above, translates Live→merged text into
+sequential OT ops, and waits for Overleaf's applied-update event. It then reads the document back
+and verifies the final text—and, in suggestion mode, tracked-change ids—before updating the saved
+base. Accept and reject operations verify that the requested ids disappeared; rejection also
+verifies the exact planned text. Push, accept/reject, comment, and reply fail loudly when their
+readback cannot prove success.
 
 ## ⚠️ Disclaimer