wenay-common2 1.0.62 → 1.0.64

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. package/CLAUDE.md +63 -0
  2. package/README.md +6 -322
  3. package/doc/NAMING_RENAMES.md +36 -0
  4. package/doc/RECOMMENDATIONS.md +56 -0
  5. package/doc/REPLAY-PLAN.md +81 -0
  6. package/doc/changes/1.0.63.md +8 -0
  7. package/doc/changes/1.0.64.md +10 -0
  8. package/doc/changes/README.md +8 -0
  9. package/doc/wenay-common2-rare.md +522 -0
  10. package/doc/wenay-common2.md +334 -0
  11. package/lib/Common/Observe/index.d.ts +2 -1
  12. package/lib/Common/Observe/index.js +2 -1
  13. package/lib/Common/Observe/reactive.d.ts +21 -0
  14. package/lib/Common/Observe/reactive.js +374 -0
  15. package/lib/Common/Observe/reactive2.d.ts +1 -21
  16. package/lib/Common/Observe/reactive2.js +15 -372
  17. package/lib/Common/Observe/store-manager.d.ts +123 -0
  18. package/lib/Common/Observe/store-manager.js +227 -0
  19. package/lib/Common/Observe/store-offline.d.ts +10 -10
  20. package/lib/Common/Observe/store-offline.js +2 -2
  21. package/lib/Common/Observe/store.d.ts +1 -1
  22. package/lib/Common/Observe/store.js +13 -13
  23. package/lib/Common/async/promiseProgress.d.ts +2 -2
  24. package/lib/Common/events/Listen.d.ts +129 -1
  25. package/lib/Common/events/Listen.js +243 -15
  26. package/lib/Common/events/Listen3.d.ts +1 -129
  27. package/lib/Common/events/Listen3.js +15 -243
  28. package/lib/Common/events/SocketBuffer.d.ts +4 -4
  29. package/lib/Common/events/SocketServerHook.d.ts +2 -2
  30. package/lib/Common/events/replay-conflate.d.ts +2 -2
  31. package/lib/Common/events/replay-conflate.js +2 -2
  32. package/lib/Common/events/replay-history.d.ts +1 -1
  33. package/lib/Common/events/replay-listen.d.ts +9 -9
  34. package/lib/Common/events/replay-listen.js +4 -4
  35. package/lib/Common/events/replay-wire.d.ts +1 -1
  36. package/package.json +9 -2
  37. package/rpc.md +293 -0
package/CLAUDE.md ADDED
@@ -0,0 +1,63 @@
1
+ # Code style (author preferences)
2
+
3
+ Write new code in this style by default — no need to ask.
4
+
5
+ ## Prohibited
6
+ - Never write new logic without first checking if it already exists in the codebase
7
+ - Never change public interfaces without explicit discussion
8
+
9
+ ## Architecture
10
+ - **Closure factories instead of classes.** `function createX(deps) { ... return { ... } }`.
11
+ Keep state in the closure; avoid classes and `this`.
12
+ - **DI at the boundary, closures inside.** A *reusable / exported* factory takes its
13
+ dependencies as an explicit `deps` object, so it can move to its own file unchanged.
14
+ *Inside* a factory, nesting is normal and encouraged — inner factories and helper
15
+ functions freely close over the enclosing scope (that captured state is the whole point
16
+ of closures). Factories don't have to be flat.
17
+ - A factory's returned object is a namespace of functions (functional style,
18
+ an "object with meaning"). Split the return **by audience** when there's more than one —
19
+ `control` (what goes *into* the owning unit) vs `api` (what's exposed *outward*). The
20
+ number of facades follows the need: **one** flat object when there's a single audience,
21
+ **two** (`control`/`api`) for the common in/out split, **more** when distinct consumers
22
+ warrant it (e.g. `control` / `api` / `debug`). Don't force two if one is clearer.
23
+ - **Layering — separate resource / utility / business logic.** Keep the *resource* part
24
+ (sockets, adapters, connections, stores) apart from *utilities* (pure, reusable helpers)
25
+ and from *business logic*. Business logic is **local**: each layer has its own — the
26
+ API-description layer carries the business rules for that API; a higher layer carries the
27
+ business rules of *using* those APIs. Wherever it lives, mark it clearly (section dividers).
28
+ Utility-leaning functions are better *extracted* (own function / file) so they don't blur
29
+ into business logic.
30
+ - **Multi-level facades are fine** — `listen` (or other callbacks) may be installed at
31
+ any nesting depth. Closures are fine. Inner factories and captured state are encouraged.
32
+ - **Expose callbacks outward.** `listen` streams that callers will subscribe to belong in
33
+ the `api` surface — prefer over-exposing event streams to hiding them.
34
+ - Registries/tables — `as const` + inferred literal types (type-safe).
35
+
36
+ ## Syntax
37
+ - **`==` / `!=`**, not `===` / `!==` (unless strict comparison is genuinely needed).
38
+ - **Don't annotate function return types** (`: Promise<void>`, `: number`, etc.) — let them
39
+ be inferred. Exception: when the type genuinely helps the reader or narrows inference.
40
+ - Single quotes, 4-space indent, no semicolons.
41
+ - **Named functions over anonymous arrows for anything non-trivial.** A named `function foo()`
42
+ shows up by name in stack traces / logs; an anonymous arrow is `<anonymous>`. So: real logic,
43
+ anything that can throw, async handlers, event callbacks → name them. Trivial one-liners where
44
+ an error is impossible (simple getters, mappers, predicates) — arrow is fine, no need to fuss.
45
+
46
+ ## Types
47
+ - Union and primitive aliases — `t` prefix: `tNum`, `tSide`, `tOrderId`.
48
+ - Generic parameters — uppercase `T`, `K`, `Cb`.
49
+ - Derive public types from the implementation: `type X = ReturnType<typeof createX>`.
50
+
51
+ ## Comments
52
+ - Section dividers like `// ===...===` with a block heading.
53
+ - Explain "why", not "what". Keep them short.
54
+ ## Documentation and release notes
55
+
56
+ - `README.md` is navigation only. Do not put API guides, examples, or long explanations there.
57
+ - The brief public surface lives in `doc/wenay-common2.md`.
58
+ - The extended/rare public surface lives in `doc/wenay-common2-rare.md`.
59
+ - Naming migrations live in `doc/NAMING_RENAMES.md`.
60
+ - Recent changes live in `doc/changes/` as one markdown file per published version, named `<version>.md`.
61
+ - Every release/change intended for publication must add or update the current version file in `doc/changes/` with a short commit-style summary of what changed.
62
+ - Keep only the latest 10 version files in `doc/changes/`; delete older version files when adding a new one.
63
+ - Do not publish until brief docs, rare docs, and the current version change file match the code being published.
package/README.md CHANGED
@@ -1,325 +1,9 @@
1
1
  # wenay-common2
2
2
 
3
- A set of common utilities and components for TypeScript/Node.js projects.
3
+ ## Documentation
4
4
 
5
- ## 📦 Library Structure
6
-
7
- ### 🔧 **core/** — Base Utilities
8
- - **`common.ts`** deep/shallow cloning and comparison, binary search (`BSearch*`), number conversion, timers, mutex, helper structures (`MyMap`, `StructMap`), clipboard handling
9
- - **`BaseTypes.ts`** `Immutable`/`Mutable` types, `const_Date`, type utilities (`PickTypes`, `OmitTypes`, `KeysByType`)
10
- - **`Decorator.ts`** — function decorators/transformers (before/after call, result processing)
11
- - **`MemoFunc.ts`** — memoization with TTL and limits, LRU eviction
12
-
13
- ### 📊 **data/** — Data Structures and Serialization
14
- - **`List.ts`** — doubly linked list `CList` with immutable interfaces
15
- - **`ListNodeAnd.ts`** — alternative doubly linked list nodes with a sentinel element
16
- - **`ByteStream.ts`** — binary serialization: `ByteStreamW` (write) and `ByteStreamR` (read), support for numeric types and arrays
17
- - **`objectPath.ts`** — working with object paths (`objectGetValueByPath`, `objectSetValueByPath`, iteration via `iterateDeepObjectEntries`)
18
-
19
- ### ⏱️ **async/** — Async Utilities
20
- - **`waitRun.ts`** — throttle/debounce (`enhancedWaitRun`), async queues (`createAsyncQueue`, `queueRun`), task queue with readiness control (`createTaskQueue`)
21
- - **`promiseProgress.ts`** — processing an array of promises with success/error subscriptions
22
- - **`createIterableObject.ts`** — proxy for iterable objects (readonly/read-write)
23
-
24
- ### 🔔 **events/** — Events and Subscriptions
25
- - **`Listen.ts`** — subscription system (`listen`, `createListen`), `isListenCallback` check
26
- - **`event.ts`** — event handler collections (`CObjectEventsArr`, `CObjectEventsList`)
27
- - **`SocketBuffer.ts`** — socket callback buffering (`socketBuffer`, `listenSnapshot`)
28
- - **`SocketServerHook.ts`** — tag-based subscription wrappers for socket communication (`SocketServerHook`, `WebSocketServerHook`)
29
- - **`joinListens.ts`** — merging multiple subscription streams into one (`joinListens`)
30
- - **`listen-socket.ts`** — bridge between event system and RPC (`listenSocket`, `listenSocketFirst`, `listenSocketAll`, `listenSocketSmart`)
31
-
32
- ### 🌐 **rpc/** — RPC System
33
- **Core:**
34
- - **`rpc-protocol.ts`** — RPC protocol (packet types `Pkt`, `SocketTmpl`)
35
- - **`rpc-client.ts`** — RPC client (`createRpcClient`, `ClientAPI`)
36
- - **`rpc-server.ts`** — RPC server (`createRpcServer`, hooks `PromiseServerHooks`)
37
- - **`rpc-walk.ts`** — structure traversal (callback serialization, `pack`/`unpack`, `resolveCA`)
38
- - **`rpc-limits.ts`** — security limits (`RpcLimits`, `PayloadLimitError`, `isSafeKey`)
39
- - **`rpc-dynamic.ts`** — dynamic object marking (`noStrict`, `isNoStrict`)
40
- - **`id-pool.ts`** — reusable ID pool (`createIdPool`)
41
-
42
- **Automation:**
43
- - **`rpc-client-auto.ts`** — client with automatic event subscription (`createRpcClientAuto`, modes `smart`/`first`/`all`)
44
- - **`rpc-server-auto.ts`** — server with automatic subscription handling (`createRpcServerAuto`)
45
- - **`listen-deep.ts`** — deep object transformation for subscriptions (`deepListenFirst`, `deepListenAll`, `deepListenSmart`)
46
-
47
- ### 📡 **network/** — Network Utilities
48
- - **`WebHook3.ts`** — webhook server/client on Express
49
- - **Server:** `createWebhookServer({ authToken, port, app? })`
50
- - **Client:** `createWebhookClient({ serverUrl, clientPort, authToken, autoRenew? })`
51
- - Support for legacy (`tag`) and new (`tags`) format in `subscribers.json`
52
-
53
- ### ⏰ **time/** — Time Handling
54
- - **`Time.ts`** — timeframes (`TF`), periodics (`Period`), time formatting, date conversion (`timeToStr_*`, `timeLocalToStr_*`)
55
- - **`funcTimeWait.ts`** — request time tracking by keys (`funcTimeW`, `FuncTimeWait`)
56
-
57
- ### 🎨 **utils/** — Helper Utilities
58
- - **`Color.ts`** — color handling (`ColorString`, `rgb`, generators `colorGenerator`, similarity check `isSimilarColors`)
59
- - **`Math.ts`** — Pearson correlation calculation (`CorrelationRollingByBuffer`)
60
- - **`isProxy.ts`** — Proxy object detection (Node.js + browser)
61
- - **`fsKeyVolume.ts`** — file system key-value storage (`saveKeyValue`)
62
- - **`inputAutoStep.ts`** — automatic step control for `<input>` (`SetAutoStepForElement`)
63
- - **`node_console.ts`** — enhanced console output with source maps (clickable links in IDE)
64
-
65
- ---
66
-
67
- ## 🚀 Quick Start
68
-
69
- ### Installation
70
- ```bash
71
- npm install wenay-common2
72
- ```
73
-
74
- Here is a comprehensive guide in a concise style. I've taken into account the architecture, backend, frontend (with the new `Hub` pattern), serialization nuances, limits, and hooks.
75
-
76
- # wenay-common2 RPC: Complete Guide
77
-
78
- Bidirectional, strongly-typed RPC protocol over sockets (Socket.IO or similar).
79
- **Essence:** Server exposes a nested JS object $\to$ Client receives a typed proxy.
80
-
81
- ---
82
-
83
- ## 1. Architecture and Limitations
84
-
85
- * **Multiplexing:** A single physical socket hosts independent channels (`socketKey`), each with its own API object.
86
- * **Data Types:** Works with JSON-compatible data plus `Date`, `Map`, `Set`, `RegExp`, and `BigInt`. Class instances are sent as plain enumerable object data; methods/prototypes are not preserved.
87
- * **Security (RpcLimits):** Server is protected from DDoS attacks. Strict limits on: `maxDepth`, `maxKeys`, `maxArrayLen`, `maxStringLen`, `maxCallbacks`. Exceeding throws `PayloadLimitError`.
88
-
89
- ---
90
-
91
- ## 2. Server (Backend)
92
-
93
- ### 2.1 Socket Connection
94
- ```typescript
95
- import { createRpcServerAuto, listen } from "wenay-common2";
96
-
97
- io.sockets.on('connection', (socket) => {
98
- // 1. Create unsubscribe trigger for memory cleanup
99
- const [stop, listenStop] = listen<[]>();
100
- socket.on('disconnect', stop);
101
-
102
- // 2. Initialize RPC channel on this socket
103
- createRpcServerAuto({
104
- socket,
105
- socketKey: "mainAPI", // Channel identifier
106
- object: buildFacade(client), // Target API object
107
- disconnectListen: listenStop, // Auto-unsubscribe from Listen on disconnect
108
- debug: process.env.DEV, // Packet logging
109
- });
110
- });
111
- ```
112
-
113
- ### 2.2 Building API Object (Facade)
114
- The object is traversed by the server to build a "Schema" that is sent to the client.
115
- ```typescript
116
- import { noStrict, listen } from "wenay-common2";
117
-
118
- // Create pub/sub event system
119
- const [sendEvent, listenEvent] = listen<[string]>();
120
-
121
- export function buildFacade(client) {
122
- const role = (...roles) => hasRole(client, roles) ? true : null;
123
-
124
- return {
125
- // 1. Regular method
126
- ping: () => "pong",
127
-
128
- // 2. Nested namespaces + Role model
129
- // If role() returns null, the method won't be sent to the client (returns null in schema)
130
- admin: {
131
- deleteUser: role("admin") && ((id) => db.delete(id)),
132
- },
133
-
134
- // 3. Dynamic objects (Proxy, ORM)
135
- // Wrap in noStrict so the server doesn't try to read keys.
136
- // Client will work with it in "blind" mode (without schema).
137
- dbRef: noStrict(getProxyDb()),
138
-
139
- // 4. Events
140
- // Client will receive a Listen surface: .on(cb), .once(cb), .close()
141
- events: { listenEvent },
142
-
143
- // 5. Method with callback in arguments
144
- // Callback lives ONLY while await is executing! After return, client deletes it.
145
- stream: async (cb: (chunk: number) => void) => {
146
- for(let i=0; i<10; i++) { cb(i); await sleep(50); }
147
- return "done";
148
- }
149
- };
150
- }
151
- ```
152
-
153
- ### 2.3 Server Hooks (Interceptors)
154
- Use hooks to validate incoming packets.
155
- ```typescript
156
- createRpcServerAuto({
157
- /*...*/
158
- hooks: {
159
- onRequest: async ({ key, request, fnName, fn }) => {
160
- // Return false to block the call
161
- return true;
162
- },
163
- onInvalid: ({ reason, key, request }) => {
164
- console.warn(`RPC Attack/Error [${reason}]:`, key);
165
- }
166
- }
167
- })
168
- ```
169
-
170
- ---
171
-
172
- ## 3. Client (Frontend)
173
-
174
- **Hub Pattern:** The frontend library doesn't depend on `socket.io-client`. Developer injects the socket factory into `createRpcClientHub`.
175
-
176
- ### 3.1 Hub Initialization
177
- ```typescript
178
- import { io } from "socket.io-client";
179
- import { createRpcClientHub, rpc } from "wenay-common2";
180
- import type { MainFacade } from "../server/facade";
181
-
182
- export const Api = createRpcClientHub(
183
- // 1. Socket factory (DI)
184
- (token) => io("http://localhost:4021", {
185
- transports: ["websocket"],
186
- query: token ? { token } : {}
187
- }),
188
-
189
- // 2. Channel registration
190
- // rpc() accepts Facade type. Property name ("mainAPI") becomes socketKey.
191
- (rpc) => ({
192
- mainAPI: rpc<MainFacade>(),
193
- })
194
- );
195
- ```
196
-
197
- ### 3.2 Connection Lifecycle
198
- ```typescript
199
- // Listen to statuses
200
- Api.onConnect((count) => console.log(`Socket connected (attempt ${count})`));
201
-
202
- // Initiate connect. Creates socket, all channels automatically start.
203
- await Api.connect("USER_SECRET_TOKEN");
204
-
205
- // Disconnect
206
- // await Api.connect(null);
207
- ```
208
-
209
- ### 3.3 Call Modes
210
- Access API channel: `Api.facade.mainAPI`. Hub initializes all channels, so schema loads automatically.
211
-
212
- ```typescript
213
- const api = Api.facade.mainAPI;
214
-
215
- // Wait for schema (REQUIRED before UI render)
216
- await api.ready();
217
-
218
- // --- 1. STRICT (Recommended) ---
219
- // Safe call. Use `?.` since method may be `null` (closed by roles).
220
- // If method is closed, returns `undefined` without sending network request.
221
- const res = await api.strict.admin?.deleteUser?.(5);
222
-
223
- // --- 2. FUNC (Standard) ---
224
- const res2 = await api.func.ping();
225
-
226
- // --- 3. PIPE (Pipeline) ---
227
- // Entire chain goes to server in ONE network packet.
228
- const data = await api.pipe.dbRef.users.find(1).getName();
229
-
230
- // --- 4. SPACE (Fire-and-Forget) ---
231
- // Doesn't wait for response, Promise resolves immediately.
232
- api.space.admin.logAction("clicked");
233
- ```
234
-
235
- ### 3.4 Client Subscriptions (Listen)
236
- `createRpcServerAuto` exposes server `listen` values as RPC Listen nodes. New code uses `on`/`once` and keeps the returned `off` handle. For TypeScript, project `client.func` to `DeepSocketListen<ServerFacade>`; this mirrors the runtime shape and keeps event argument types.
237
- ```typescript
238
- import type { DeepSocketListen } from "wenay-common2";
239
-
240
- function webListen<T extends object>(client: { func: unknown }) {
241
- return client.func as DeepSocketListen<T>;
242
- }
243
-
244
- const events = webListen<MainFacade>(api).events;
245
-
246
- // Subscribe
247
- const off = events.listenEvent.on((msg) => {
248
- console.log("Push from server:", msg);
249
- });
250
-
251
- // Unsubscribe. The handle is callable and also thenable.
252
- off();
253
- // await off; // waits until the stream ends
254
-
255
- // One event, then automatic unsubscribe
256
- const done = events.listenEvent.once((msg) => {
257
- console.log("First push:", msg);
258
- });
259
- await done;
260
- ```
261
-
262
- Compatibility names `.callback(cb)`, `.removeCallback()`, and `.unsubscribe()` still exist for old clients, but they are not the recommended API.
263
-
264
-
265
- ### 3.5 Request Management and Debug
266
- Each facade has a system object `api` for low-level control, as well as a couple of methods in the root:
267
-
268
- ```typescript
269
- const { api, abortAll, schema } = Api.facade.mainAPI;
270
-
271
- // --- 1. Monitoring and Debug ---
272
- api.pending(); // Current number of pending responses (Promises)
273
- api.callbacks(); // Current number of live callback ids in memory
274
- api.log(true); // Enable logging of all incoming/outgoing packets to console
275
-
276
- // --- 2. Targeted cleanup (inside .api) ---
277
- api.clearPromises(true); // Reject (cancel) all current requests
278
- api.clearCallbacks(); // Force clear all callbacks
279
- api.remove(myFunc); // Force-release a specific callback id (alias: .end)
280
-
281
- // --- 3. Global facade methods ---
282
- abortAll("User logout"); // Hard reset: reject all promises with RPC_ABORT error + clear all callback ids
283
- const map = schema(); // Get raw schema tree (MAP) sent by server
284
- ```
285
-
286
-
287
- ---
288
-
289
- ## 4. Advanced Features
290
-
291
- ### 4.1 Listen Argument Interception Modes
292
- When using events, the client auto-handler (`mode: "smart"`) by default flexibly adapts arguments. If server sends one argument — it comes as a value, if multiple — as an array.
293
- If you create a client manually without Hub, you can set a strict mode:
294
- ```typescript
295
- // "first" — listener always receives only the first argument
296
- // "all" — listener always receives all arguments
297
- // "smart" — (default) auto-detection
298
- const autoApi = createRpcClientAuto(api.func, { mode: "first" });
299
- ```
300
-
301
-
302
- ### 4.2 Manual Callback Termination from Server
303
- This is a low-level escape hatch for callbacks passed as ordinary function arguments. For Listen subscriptions prefer `off()`/`.once()`.
304
- ```typescript
305
- import { endCallback } from "wenay-common2";
306
-
307
- async function myMethod(cb: (data: any) => void) {
308
- cb("chunk 1");
309
- endCallback(cb); // Alias: rpcEndCallback. Sends "___STOP" to client.
310
- // Client callback id is deleted, subsequent cb() calls won't go anywhere.
311
- }
312
- ```
313
-
314
- ### 4.3 Call / Apply Support on Client
315
- The client proxy can transparently handle standard JS `call` and `apply` calls. Server correctly normalizes them ("collapses" the path):
316
- ```typescript
317
- // Both variants correctly call `api.users.create("Ivan")` on server
318
- await api.func.users.create.call(null, "Ivan");
319
- await api.func.users.create.apply(null, ["Ivan"]);
320
- ```
321
-
322
-
323
- ### 4.4 Transparent PIPE Request Transit
324
- For microservice architecture. If your Node server itself is a client of another RPC node (via `pipe`), it can "pass through" the remainder of the pipe chain further using `__executeRemainingPipe`, without waiting for an intermediate response.
325
- This covers the important RPC mechanics (Listen `on/once/off`, `endCallback`, `call/apply`, `pipe-transit`) while maintaining readability.
5
+ - Brief API cheat sheet: [`doc/wenay-common2.md`](doc/wenay-common2.md)
6
+ - Extended API cheat sheet: [`doc/wenay-common2-rare.md`](doc/wenay-common2-rare.md)
7
+ - Naming migrations: [`doc/NAMING_RENAMES.md`](doc/NAMING_RENAMES.md)
8
+ - Recent changes: [`doc/changes/`](doc/changes/)
9
+ - Project rules for AI/code maintenance: [`CLAUDE.md`](CLAUDE.md)
@@ -0,0 +1,36 @@
1
+ # Карта переименований
2
+
3
+ Breaking migration: старые имена не оставляем алиасами.
4
+
5
+ | Было | Стало |
6
+ | --- | --- |
7
+ | `UseListen` | `listen` |
8
+ | `UseListenStore` | `listenStore` |
9
+ | `UseListen2` | `slimListen` |
10
+ | `toListen2` | `toSlimListen` |
11
+ | `UseListenTransform` | `mapListen` |
12
+ | `funcListenCallbackBase` | `createListen` |
13
+ | `funcListenCallbackFast` | `createFastListen` |
14
+ | `funcListenCallbackStore` | `createStoreListen` |
15
+ | `funcListenCore` | `createListenCore` |
16
+ | `addListen` | `on` |
17
+ | `removeListen` | `off` |
18
+ | `eventClose` | `onClose` |
19
+ | `removeEventClose(cb)` | `const off = onClose(cb); off()` |
20
+ | `addListenClose` | `closeOn` |
21
+ | `tSubHandle` | `SubscriptionHandle` |
22
+ | `PromiseArrayListen` | `promiseProgress` |
23
+ | `listenOk` / `listenError` | `onOk` / `onError` |
24
+ | `promise.all()` / `promise.allSettled()` | `all()` / `allSettled()` |
25
+ | `getData()` / `status()` | `items()` / `stats()` |
26
+ | `realSocket2` | `SocketSource` |
27
+ | `getTypeCallback` | `SocketPayload` |
28
+ | `socketBuffer3` | `socketBuffer` |
29
+ | `funcListenCallbackSnapshot` | `listenSnapshot` |
30
+ | `createRpcServerAuto2` | `createRpcServerAutoDetect` |
31
+ | `UseReplayListen` | `replayListen` |
32
+ | `ListenNext` | root exports from `wenay-common2` (`listen`, `listenStore`, `slimListen`) |
33
+ | `wenay-common2/listen2` | removed; use root exports from `wenay-common2` |
34
+ | `ObserveAll2` | `Observe` |
35
+ | `wenay-common2/observe-all2` | `wenay-common2/observe` |
36
+ | `src/Common/ObserveAll2` | `src/Common/Observe` |
@@ -0,0 +1,56 @@
1
+ # Recommendations — Listen / Observe / RPC integration
2
+
3
+ Status refreshed 2026-07-08. This document is now a current checklist, not the
4
+ old 2026-07-02 migration scratchpad. Public code should import the canonical
5
+ surfaces (`wenay-common2`, `wenay-common2/observe`, `wenay-common2/replay`, or
6
+ `src/Common/events/Listen` inside the repo). Numbered implementation artifacts
7
+ such as `Listen3` and `reactive2` are compatibility shims only.
8
+
9
+ ## 1. Naming cleanup
10
+
11
+ - **DONE** `src/Common/events/Listen.ts` is the real implementation file.
12
+ `Listen3.ts` remains only as an internal deep-import shim.
13
+ - **DONE** `src/Common/Observe/reactive.ts` is the real reactive implementation.
14
+ `reactive2.ts` remains only as an internal deep-import shim.
15
+ - **DONE** the oracle folder is `observe/`, not `observable2/`.
16
+ - **RULE** new code and docs should not introduce `UseListen`, `Listen3`,
17
+ `reactive2`, or `observable2` outside `NAMING_RENAMES.md`, historical `doc/changes/` notes, or shim comments.
18
+
19
+ ## 2. Observe store status
20
+
21
+ - **DONE** circular `Map`/`Set` snapshots are guarded by `seen` before container
22
+ traversal.
23
+ - **DONE** `createStoreMirror.sync` reports failed re-pulls through
24
+ `StoreSyncOpts.onError`; initial pull failure still rejects the awaited sync.
25
+ - **DONE** path cache identity separates dotted keys and distinct symbols.
26
+ - **DONE** brief/rare docs describe the store layer as separate from the coarse
27
+ reactive core.
28
+ - **OPEN** `_nodeCache` and `_counts` are unbounded for dynamic path spaces.
29
+ Either document bounded-path expectations for applications or add eviction.
30
+ - **OPEN** `schedule` / `createDrained` in `store.ts` still duplicate scheduler
31
+ logic from `reactive.ts`; extract only if more local schedulers appear.
32
+
33
+ ## 3. Store manager direction
34
+
35
+ The universal manager should stay above `Observe.createStore` / replay / offline.
36
+ It should not make the store core know about products, routes, user habits, or
37
+ large-resource policy. The manager owns resource declarations, priority, tags,
38
+ usage scoring, explicit-only gates, and start/stop lifecycle.
39
+
40
+ Current implementation target: `Observe.createStoreManager` with `managedStore`
41
+ builders for `mirror`, `replay`, and `offline` resources.
42
+
43
+ ## 4. RPC current over wire
44
+
45
+ Still open. The client dedup key already includes non-function args, but the
46
+ server subscription path does not yet whitelist and forward `{current:true}` to
47
+ Listen store decorators. Do not document `{current:true}` over RPC as supported
48
+ until server forwarding and late-join replay are implemented together.
49
+
50
+ ## 5. Suggested order
51
+
52
+ 1. Keep naming cleanup green with `rg` before release.
53
+ 2. Add tests for `createStoreManager` planning, explicit gates, mirror sync, and
54
+ offline/replay startup.
55
+ 3. Update brief and rare docs only after tests pass.
56
+ 4. Bump patch version, build, then publish from `dist`.
@@ -0,0 +1,81 @@
1
+ # Snapshot + Sequenced Delta Line — universal state sync
2
+
3
+ One pattern for everything: **keyframe (snapshot) + numbered line of deltas + recovery via fresh keyframe**.
4
+ Payload-agnostic: store patches, ticks, any events. Same figure as market-data feeds
5
+ (incremental + snapshot channel), DB replication (basebackup + WAL), video (I/P frames), Kafka.
6
+
7
+ ## Invariants
8
+
9
+ - **seq is the coordinate, time is an attribute.** Every journaled event: `{seq, ts, event}`.
10
+ seq = monotonic counter → strict order, gap detection, exact replay→live handover.
11
+ Time is for humans only: "rewind to 12:00" resolves to nearest keyframe ≤ ts → its seq.
12
+ - **Keyframe = state as of seq S.** Rule of the whole system:
13
+ `keyframe(S) + events S+1…K = exact state at K`.
14
+ - **Memory stays external** (lambda providers), decorators own no data they don't have to.
15
+ - **Additive only.** The canonical Listen surface is untouched; new surface = decorator + opts.
16
+ - The live store is never reset: archiving = take a labeled copy, keep mutating the same store.
17
+
18
+ ## The killer property (drives everything)
19
+
20
+ A lagging/late/reconnecting consumer never gets a queue backlog — it gets a **fresh keyframe
21
+ + the line from there**. This is both the catch-up mechanism AND the backpressure policy
22
+ (conflation + snapshot recovery). No unbounded buffering, ever.
23
+
24
+ ## Layers
25
+
26
+ ### A. Listen replay decorator — `withReplayListen` (universal, payload-agnostic)
27
+
28
+ Decorator over `ListenApi`, exact shape of `withStoreListen`:
29
+
30
+ ```
31
+ withReplayListen(base, {
32
+ current, // () => keyframe — external memory lambda (exists: layer 3)
33
+ history?: number, // ring buffer size N (internal) …
34
+ getSince?: (seq) => events[], // …or external journal lambda (memory outside — preferred)
35
+ })
36
+ ```
37
+
38
+ Subscription modes:
39
+ - `on(cb)` — live only (unchanged)
40
+ - `on(cb, {current: true})` — keyframe + live (unchanged, layer 3)
41
+ - `on(cb, {since: seq})` — catch-up: replay journal from seq+1, queue live events by seq
42
+ during replay, dedup by seq, seamless switch to live. seq evicted from journal →
43
+ fallback: keyframe + live.
44
+
45
+ The handover (replay→live without gap or dup) is the ONLY subtle code in the plan.
46
+ Everything else is mechanics.
47
+
48
+ ### B. Observe store integration
49
+
50
+ - `exposeStore`: patch journal — every emitted `StorePatch` gets seq; keep last N (ring).
51
+ `snapshot()` responses carry the seq they correspond to.
52
+ - `createStoreMirror`: sync = "I have seq K" → tail of patches, or snapshot(S) + live from S+1.
53
+ Reconnect stops costing a full snapshot when the tail suffices.
54
+ - Uses layer A; store adds nothing but "patch as the event type".
55
+
56
+ ### C. History storage (optional, lazy, fully external)
57
+
58
+ For seek into the past — not needed for live sync:
59
+ - periodic archiving: keyframe every N events OR T seconds, whichever first (GOP tradeoff:
60
+ dense = fast seek / more space, sparse = cheap / longer replay);
61
+ - providers: `getKeyframe(seqOrTs)`, `getEvents(from, to)` — file/DB/anything, behind lambdas;
62
+ - a history reader is the SAME subscriber interface: subscribe `{since}` → archive replay →
63
+ journal catch-up → live. One mechanism for playback and live.
64
+
65
+ ### D. Transport hardening (independent add-ons, any order, later)
66
+
67
+ - **Per-client conflation**: outgoing buffer over threshold → stop deltas for that client,
68
+ mark "needs keyframe"; buffer drained → fresh snapshot + resume from current seq.
69
+ - **Serialize once, fan out**: pack a tick once, send bytes to all subscribers
70
+ (extends existing Caps.COMPACT / shape packing).
71
+ - **Coalescing**: N updates of one key while client lagged → send only the last.
72
+ - **Binary passthrough** in `rpc-walk`: TypedArray/ArrayBuffer/Buffer as leaf values
73
+ (socket.io carries binary natively; today walk() mangles them into `{0:…,1:…}`).
74
+
75
+ ## Stages
76
+
77
+ 1. `withReplayListen` decorator + seq handover (layer A) — the core, small.
78
+ 2. Patch journal in `exposeStore` + `since`-sync in mirror (layer B).
79
+ 3. Conflation + snapshot recovery on the wire (first item of D) — this is what turns
80
+ "semi-pro" into pro for fan-out.
81
+ 4. C and the rest of D — on demand.
@@ -0,0 +1,8 @@
1
+ # 1.0.63
2
+
3
+ - Add `Observe.createStoreManager` and `Observe.managedStore` resource builders for declarative mirror/replay/offline store startup.
4
+ - Add manager planning with priority, tags, usage scoring, `large`, and `explicitOnly` gates.
5
+ - Add manager lifecycle helpers: `start`, `startPlanned`, `stop`, `stopAll`, `get`, `touch`, `usage`, `statusListen`, and typed `handles`.
6
+ - Rename public-facing Observe oracle folder from `observable2` to `observe` and make `Listen.ts` / `reactive.ts` the canonical source files, leaving numbered files as compatibility shims.
7
+ - Update brief and extended API docs for the store manager and current naming.
8
+ - Publish package `wenay-common2@1.0.63`.
@@ -0,0 +1,10 @@
1
+ # 1.0.64
2
+
3
+ - Make `README.md` navigation-only and point it to the brief docs, rare docs, naming migrations, recent changes, and project rules.
4
+ - Add the `doc/changes/` release-note catalog rule to `CLAUDE.md`: one file per published version, commit-style summary, latest 10 versions only.
5
+ - Add the recent changes catalog README and keep the current release notes under `doc/changes/`.
6
+ - Package documentation, `CLAUDE.md`, and `rpc.md` into `dist` so README links work in the npm package.
7
+ - Restore the `noStrict` dynamic-map contract in the extended docs after removing a duplicated store-manager block.
8
+ - Move regression oracles to the canonical `observe/reactive` names after the `observable2/reactive2` rename.
9
+ - Normalize npm repository metadata.
10
+ - Publish package `wenay-common2@1.0.64`.
@@ -0,0 +1,8 @@
1
+ # Recent changes
2
+
3
+ This directory keeps one file per published version.
4
+
5
+ Rules:
6
+ - file name: `<version>.md`, for example `1.0.63.md`;
7
+ - each file contains a short commit-style summary of what changed;
8
+ - keep only the latest 10 version files, deleting older version files when adding a new one.