@ibgib/core-gib 0.1.59 → 0.1.61

package/src/sync/docs/security.md

@@ -184,6 +184,27 @@ Context ibgib now has a clean, consistent pair:
 
 ---
 
+## respec-gib Testing Framework
+
+Because `ibgib` has unique execution environments and a high need for custom asynchronous block handling, it uses a custom testing framework located in `@ibgib/helper-gib/dist/respec-gib`.
+
+### Key Framework APIs
+* **`respecfully`**: The equivalent of `describe` or an outer test suite block. It is async and MUST be awaited.
+* **`ifWe`**: The equivalent of `it`, used for defining a specific test case block. It is async and MUST be awaited. It cannot be nested.
+* **`ifWeMight`**: A focused test case (equivalent to `fit` or `it.only`). It is async, MUST be awaited, and cannot be nested.
+* **`iReckon`**: The assertion fluent API (equivalent to `expect`). It is synchronous and should NOT be awaited. Available assertions include: `willEqual(expected)`, `isGonnaBe(expected)`, `isGonnaBeTruthy()`, `isGonnaBeFalsy()`, `isGonnaBeTrue()`, `isGonnaBeFalse()`, `isGonnaBeUndefined()`, and `includes(expected)`. You can modify assertions using `.not` and `.asTo(label)`.
+
+### Crucial Execution & Logging Nuances
+> [!IMPORTANT]
+> **File Focus vs. Code Execution**:
+> The test runner scans all test files for the presence of `ifWeMight` (or `respecfullyDear`) to determine which *files* get selected and executed. However, once a file is selected and run, **the entire file executes**.
+>
+> **The Logging Caveat**:
+> The `ifWeMight` filter **only** determines which test results are ultimately *reported* in the test summary output. This means all code and `console.log` statements in any standard `ifWe` blocks within the same file will **still execute and write to the logs**.
+> This behavior can make test logs extremely noisy and confusing. If you are focusing on a single test, you should temporarily change all other `ifWeMight` blocks to `ifWe`, and use a single `ifWeMight` block or comment out other logs/tests to keep console output readable.
+
+---
+
 ## Implementation Plan
 
 Each phase below is a discrete, verifiable step. We proceed **one checkbox at a time**, **reviewing/discussing** code/results before moving to the next. Each phase has two targets: **innerspace** (automated unit test) and **space-gib** (manual/integrated e2e).
@@ -285,29 +306,29 @@ We still call `senderCoordinator.sync(...)` — the phase focus is on what `peer
 
 **Goal**: With identity fully established and transport connected, sync a single ibgib `X` from source to dest. Verify S evolves correctly per turn (`sync` pool), and X's full dependency graph arrives on dest. After commit, both sender and receiver durable spaces must contain: the same evolved I, the full S dependency graph, and the full X dependency graph.
 
-#### Phase 3A — Innerspace unit test (`sync-withid.fullsync.respec.mts`)
+#### Phase 3A — Innerspace unit test (`sync-withid.pingpong.respec.mts`)
 
-- [ ] Create `sync-withid.fullsync.respec.mts`
-  - [ ] Create `r1_alpha_v0_source` via `TestTransformer`
-  - [ ] Call `senderCoordinator.sync({ domainIbGibs: [alpha], senderIdentity, senderSecret, ... })`
-  - [ ] `ifWeMight` — `"alpha dep graph matches on source and dest"`: use `getDependencyGraph` + `graphsAreEquivalent`
-  - [ ] `ifWeMight` — `"sessionIdentity evolved the expected number of times"`: assert `S.data.n` equals the known deterministic turn count
-    - [ ] 🔲 **TODO**: Manually inspect the full session keystone graph after a first run and count the exact number of outgoing turns (Init + Delta(s) + Commit), then hard-code that number into this assertion
-  - [ ] `ifWeMight` — `"sender durable space has I (evolved frame)"`: assert I1 addr is in `sourceSpace`
-  - [ ] `ifWeMight` — `"receiver durable space has I (evolved frame)"`: assert I1 addr is in `destSpace`
-  - [ ] `ifWeMight` — `"sender durable space has full S dep graph"`: use `getDependencyGraph` on S in `sourceSpace`
-  - [ ] `ifWeMight` — `"receiver durable space has full S dep graph"`: use `getDependencyGraph` on S in `destSpace`
+- [x] Create `sync-withid.pingpong.respec.mts`
+  - [x] Create `r1_alpha_v0_source` (represented by `xStone` test domain ibgib)
+  - [x] Call `senderCoordinator.sync({ domainIbGibs: [alpha], senderIdentity, senderSecret, ... })`
+  - [x] `ifWeMight` — `"alpha dep graph matches on source and dest"`: use `getDependencyGraph` + `graphsAreEquivalent`
+  - [x] `ifWeMight` — `"sessionIdentity evolved the expected number of times"`: assert `S.data.n` equals the known deterministic turn count
+    - [x] ☑ **TODO**: Manually inspect the full session keystone graph after a first run and count the exact number of outgoing turns (Init + Delta(s) + Commit), then hard-code that number into this assertion
+  - [x] `ifWeMight` — `"sender durable space has I (evolved frame)"`: assert I1 addr is in `sourceSpace`
+  - [x] `ifWeMight` — `"receiver durable space has I (evolved frame)"`: assert I1 addr is in `destSpace`
+  - [x] `ifWeMight` — `"sender durable space has full S dep graph"`: use `getDependencyGraph` on S in `sourceSpace`
+  - [x] `ifWeMight` — `"receiver durable space has full S dep graph"`: use `getDependencyGraph` on S in `destSpace`
 
 #### Phase 3B — Space-Gib integrated (manual)
 
-- [ ] Trigger a sync of a known test ibgib via `dev-tools.mts` (add "sync - fullsync" button to a new row)
-- [ ] Confirm S evolves on each outgoing turn (inspect frame `data.n` in debugger)
-- [ ] Confirm X's full dependency graph is present on the client after commit
-- [ ] Confirm X's full dependency graph is present on the server after commit
-- [ ] Confirm S's full dependency graph is present on the client after commit
-- [ ] Confirm S's full dependency graph is present on the server after commit
-- [ ] Confirm I's evolved frame (I1) is present on the client after commit
-- [ ] Confirm I's evolved frame (I1) is present on the server after commit
+- [x] Trigger a sync of a known test ibgib via `dev-tools.mts` (add "sync - fullsync" button to a new row)
+- [x] Confirm S evolves on each outgoing turn (inspect frame `data.n` in debugger)
+- [x] Confirm X's full dependency graph is present on the client after commit
+- [x] Confirm X's full dependency graph is present on the server after commit
+- [x] Confirm S's full dependency graph is present on the client after commit
+- [x] Confirm S's full dependency graph is present on the server after commit
+- [x] Confirm I's evolved frame (I1) is present on the client after commit
+- [x] Confirm I's evolved frame (I1) is present on the server after commit
 
 
 ---
@@ -320,14 +341,14 @@ The sync proper will not be called. The test only cares that the pre-connect ide
 
 #### Phase 1A — Innerspace unit test
 
-- [ ] Create `sync-with-identity-basic.respec.mts` with scaffold:
+- [x] Create `sync-withid.establish.respec.mts` with scaffold:
   - `Metaspace_Innerspace` + two `InnerSpace_V1` (source/dest) + `SyncSagaCoordinator` pair + `newTestPeer()` factory
   - A `senderSecret` constant (test-only plaintext)
   - A `KeystoneService_V1` instance (new'd inline)
-- [ ] Add `respecfully` block: `"Phase 1: establishSessionIdentity"`
-  - [ ] `ifWeMight` — `"creates sessionIdentity genesis (S) locally"`: assert `S^Stjp` was generated and exists in `sourceSpace`
-  - [ ] `ifWeMight` — `"evolves senderIdentity (I → I1) with sync claim"`: assert `I1` frame has a proof whose `claim.verb === 'sync'` and `claim.target === S^Stjp`
-  - [ ] `ifWeMight` — `"posts I1 and S to destSpace (receiver)"`: assert both addrs are retrievable from `destSpace`
+- [x] Add `respecfully` block: `"Phase 1: establishSessionIdentity"`
+  - [x] `ifWeMight` — `"creates sessionIdentity genesis (S) locally"`: assert `S^Stjp` was generated and exists in `sourceSpace`
+  - [x] `ifWeMight` — `"evolves senderIdentity (I → I1) with sync claim"`: assert `I1` frame has a proof whose `claim.verb === 'sync'` and `claim.target === S^Stjp`
+  - [x] `ifWeMight` — `"posts I1 and S to destSpace (receiver)"`: assert both addrs are retrievable from `destSpace`
 
 #### Phase 1B — Space-Gib integrated (manual)
 
@@ -344,7 +365,7 @@ The sync proper will not be called. The test only cares that the pre-connect ide
 
 #### Phase 2A — Innerspace unit test
 
-- [x] Extend `sync-with-identity-basic.respec.mts` with a `respecfully` block: `"Phase 2: connect"` (Note: implemented in separate file `sync-withid.connect.respec.mts`)
+- [x] Create `sync-withid.connect.respec.mts` with a `respecfully` block: `"Phase 2: connect"`
   - [x] `ifWeMight` — `"connect completes without error"`: call `peer.connect()` and assert no exception
   - [x] `ifWeMight` — `"connect pool is depleted by exactly one challenge"`: assert `S.data.pools['connect'].used` increased by the expected count
 
@@ -362,19 +383,99 @@ The sync proper will not be called. The test only cares that the pre-connect ide
 
 #### Phase 3A — Innerspace unit test
 
-- [ ] Extend `sync-with-identity-basic.respec.mts` or create `sync-with-identity-singletimeline.respec.mts`
-  - [ ] Create `r1_alpha_v0_source` via `TestTransformer`
-  - [ ] Call `senderCoordinator.sync({ domainIbGibs: [alpha], senderIdentity, senderSecret, ... })`
-  - [ ] `ifWeMight` — `"alpha dep graph matches on source and dest"`: use `getDependencyGraph` + `graphsAreEquivalent`
-  - [ ] `ifWeMight` — `"sessionIdentity evolved once per sync turn"`: assert `S.data.n` equals number of outgoing turns
+- [x] Create `sync-withid.pingpong.respec.mts`
+  - [x] Create `r1_alpha_v0_source` via `TestTransformer` (represented by `xStone` test domain ibgib)
+  - [x] Call `senderCoordinator.sync({ domainIbGibs: [alpha], senderIdentity, senderSecret, ... })`
+  - [x] `ifWeMight` — `"alpha dep graph matches on source and dest"`: use `getDependencyGraph` + `graphsAreEquivalent`
+  - [x] `ifWeMight` — `"sessionIdentity evolved once per sync turn"`: assert `S.data.n` equals number of outgoing turns
 
 #### Phase 3B — Space-Gib integrated (manual)
 
-- [ ] Trigger a sync of a known test ibgib via `dev-tools.mts` or equivalent UI
-- [ ] Confirm S evolves on each outgoing turn (inspect frame `data.n` in debugger)
-- [ ] Confirm X's full dependency graph is present on the client after commit
-- [ ] Confirm X's full dependency graph is present on the server after commit
-- [ ] Confirm S's full dependency graph is present on the client after commit
-- [ ] Confirm S's full dependency graph is present on the server after commit
-- [ ] Confirm I's full dependency graph is present on the client after commit
-- [ ] Confirm I's full dependency graph is present on the server after commit
+- [x] Trigger a sync of a known test ibgib via `dev-tools.mts` or equivalent UI
+- [x] Confirm S evolves on each outgoing turn (inspect frame `data.n` in debugger)
+- [x] Confirm X's full dependency graph is present on the client after commit
+- [x] Confirm X's full dependency graph is present on the server after commit
+- [x] Confirm S's full dependency graph is present on the client after commit
+- [x] Confirm S's full dependency graph is present on the server after commit
+- [x] Confirm I's full dependency graph is present on the client after commit
+- [x] Confirm I's full dependency graph is present on the server after commit
+
+---
+
+### Phase 4 — Advanced Data and Conflict Scenarios
+
+**Goal**: Extend the WebSocket peer and Space-Gib interface to manually execute and verify all advanced data and conflict scenarios currently covered by the InnerSpace unit tests.
+
+> [!IMPORTANT]
+> **Guidelines & Testing Reminders**:
+> - Before starting the implementation of any automated test (A) or manual verification (B) in Phase 4, you **MUST** scan/re-scan the variable naming and structural guidelines in [testing.md](file:///c:/Users/billm/antigravity/ibgib/libs/core-gib/src/sync/docs/testing.md) to align the codebase structure and naming conventions.
+> - **Manual Testing (B) Rule**: The AI assistant **MUST NOT** use the browser subagent to perform manual browser/UI testing for any "B" manual verification phases in space-gib. The user will run all B testing themselves to conserve tokens.
+> - **Executing Focused Tests**: After changing previous test phase file's ifWeMight blocks -> ifWe to defocus that file, and after you have setup the new test file with ifWeMight blocks, use `npm run test:core-gib` to execute the new *.respec.mts file. (NOTE: If you search files for these blocks, be sure to include the open paren, e.g., "ifWeMight(", because often ifWeMight remains unused in import statements). After you have executed the tests and they pass, do NOT change it back to ifWe blocks immediately, so I can execute and look at the tests myself.
+
+For each scenario:
+1. **Automated Verification (A)**: Copy the grandfathered non-identity unit test file (e.g., `sync-innerspace.respec.mts`) to a new file named with `.withid.` in the filename (e.g., `sync-innerspace.withid.respec.mts`). Incorporate identity setup (referencing `sync-withid.pingpong.respec.mts` for setup pattern) and ensure the test runs and passes.
+   * **Test naming/structure convention**: The new `.withid.respec.mts` tests must follow the variable and structural naming guidelines outlined in [testing.md](file:///c:/Users/billm/antigravity/ibgib/libs/core-gib/src/sync/docs/testing.md).
+   * **Identity Verification Checks**: Each `.withid.respec.mts` test must verify that the session identity `S` evolved the expected number of times, the user identity `I` evolved the expected number of times on both spaces, and that the full dependency graphs for `S` and `I` are correctly synced to both spaces.
+2. **Manual Verification (B)**: Create manual websocket peer tests with corresponding buttons/UI elements integrated via Dev Tools.
+   * **Dont use browser testing**: We must minimize token use, so I will do all of the manual testing in the browser. Do not do automated browser testing in B phases.
+
+#### 1. Sync InnerSpace (Basic Push Sync)
+- [x] Phase 4.1A — Automated InnerSpace Verification (`sync-innerspace.withid.respec.mts`)
+- [x] Phase 4.1B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests basic push sync of a single timeline and its mutations from Source to Destination.
+  * Verifies that the target timeline tip and its dependency graph successfully arrive at the destination.
+
+#### 2. Sync Constants (No TJP)
+- [x] Phase 4.2A — Automated InnerSpace Verification (`sync-innerspace-constants.withid.respec.mts`)
+- [x] Phase 4.2B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests syncing "constants" (ibGibs without timelines/TJPs) and verifies that dependencies (linked constants) are resolved.
+  * Ensures idempotency/smart diffing (payloads are not re-sent if they already exist in the destination space).
+
+#### 3. Sync Destination with Partial History
+- [x] Phase 4.3A — Automated InnerSpace Verification (`sync-innerspace-partial-update.withid.respec.mts`)
+- [x] Phase 4.3B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests push synchronization when the destination already contains a partial timeline history (e.g., up to V1), but the sender has newer frames (V2).
+  * Verifies that only the delta is transmitted and correctly linked to the existing history at the destination.
+- [x] INTERSTITIAL — Refactor `dev-tools.mts` to split it into a `dev-tools` directory containing modular sub-files to reduce file size.
+
+#### 4. Sync Deep History Updates
+- [x] Phase 4.4A — Automated InnerSpace Verification (`sync-innerspace-deep-updates.withid.respec.mts`)
+- [x] Phase 4.4B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests push synchronization of a timeline tip with a deep mutation history (multiple updates).
+  * Verifies that the destination receives all intermediate frames, preserving the full timeline history and graph integrity.
+
+#### 5. Sync Destination Ahead (Remote Newer)
+- [x] Phase 4.5A — Automated InnerSpace Verification (`sync-innerspace-dest-ahead.withid.respec.mts`)
+- [x] Phase 4.5B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests synchronization when the destination (remote) actually has a newer frame (V2) than the sender's local tip (V1).
+  * Verifies that the sender learns of the remote tip and pulls it down so both client and server converge to the dest-ahead tip.
+
+#### 6. Sync Multiple Independent Timelines
+- [x] Phase 4.6A — Automated InnerSpace Verification (`sync-innerspace-multiple-timelines.withid.respec.mts`)
+- [x] Phase 4.6B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests synchronization of multiple independent timelines (e.g., Timeline A and Timeline B) in a single sync coordinator call.
+  * Verifies that both timeline tips and their respective dependency graphs are transferred and updated correctly.
+
+#### 7. Basic Divergence Conflict Resolution
+- [x] Phase 4.7A — Automated InnerSpace Verification (`sync-conflict-basic-divergence.withid.respec.mts`)
+- [x] Phase 4.7B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests divergence when source and destination edit different fields on a shared base timeline simultaneously.
+  * Verifies that optimistic merge automatically produces a merged tip (V3) containing both edits and records the graft metadata (`graftbase`, `graftorphan`).
+
+#### 8. Divergence with Related Timelines
+- [x] Phase 4.8A — Automated InnerSpace Verification (`sync-conflict-basic-multitimelines.withid.respec.mts`)
+- [x] Phase 4.8B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests advanced divergence where one of the divergent updates also references/relates a new independent timeline (Beta).
+  * Verifies that during optimistic merge, the dependent timeline is synced automatically alongside the merged tip.
+
+#### 9. Text Field Merge (LCS) Conflict Resolution
+- [x] Phase 4.9A — Automated InnerSpace Verification (`sync-conflict-text-merge.withid.respec.mts`)
+- [x] Phase 4.9B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests automatic merging of conflicting string edits within the `ibgib.data.text` field using the LCS (Longest Common Subsequence) merge algorithm.
+  * Verifies merging of simple appends (beginning vs. end) and interleaved edits in different paragraphs.
+
+#### 10. Advanced Multi-Round/Timeline Permutations
+- [x] Phase 4.10A — Automated InnerSpace Verification (`sync-conflict-adv-multitimelines.withid.respec.mts`)
+- [x] Phase 4.10B — WebSocket Peer Manual Verification (integrated via Dev Tools)
+  * Tests complex multi-round/timeline permutations with parallel divergent edits across multiple rounds.
+  * Verifies that nested or sequential merges resolve correctly using the optimistic graft strategy.