wenay-common2 1.0.62 → 1.0.64
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CLAUDE.md +63 -0
- package/README.md +6 -322
- package/doc/NAMING_RENAMES.md +36 -0
- package/doc/RECOMMENDATIONS.md +56 -0
- package/doc/REPLAY-PLAN.md +81 -0
- package/doc/changes/1.0.63.md +8 -0
- package/doc/changes/1.0.64.md +10 -0
- package/doc/changes/README.md +8 -0
- package/doc/wenay-common2-rare.md +522 -0
- package/doc/wenay-common2.md +334 -0
- package/lib/Common/Observe/index.d.ts +2 -1
- package/lib/Common/Observe/index.js +2 -1
- package/lib/Common/Observe/reactive.d.ts +21 -0
- package/lib/Common/Observe/reactive.js +374 -0
- package/lib/Common/Observe/reactive2.d.ts +1 -21
- package/lib/Common/Observe/reactive2.js +15 -372
- package/lib/Common/Observe/store-manager.d.ts +123 -0
- package/lib/Common/Observe/store-manager.js +227 -0
- package/lib/Common/Observe/store-offline.d.ts +10 -10
- package/lib/Common/Observe/store-offline.js +2 -2
- package/lib/Common/Observe/store.d.ts +1 -1
- package/lib/Common/Observe/store.js +13 -13
- package/lib/Common/async/promiseProgress.d.ts +2 -2
- package/lib/Common/events/Listen.d.ts +129 -1
- package/lib/Common/events/Listen.js +243 -15
- package/lib/Common/events/Listen3.d.ts +1 -129
- package/lib/Common/events/Listen3.js +15 -243
- package/lib/Common/events/SocketBuffer.d.ts +4 -4
- package/lib/Common/events/SocketServerHook.d.ts +2 -2
- package/lib/Common/events/replay-conflate.d.ts +2 -2
- package/lib/Common/events/replay-conflate.js +2 -2
- package/lib/Common/events/replay-history.d.ts +1 -1
- package/lib/Common/events/replay-listen.d.ts +9 -9
- package/lib/Common/events/replay-listen.js +4 -4
- package/lib/Common/events/replay-wire.d.ts +1 -1
- package/package.json +9 -2
- package/rpc.md +293 -0
package/CLAUDE.md
ADDED
|
@@ -0,0 +1,63 @@
|
|
|
1
|
+
# Code style (author preferences)
|
|
2
|
+
|
|
3
|
+
Write new code in this style by default — no need to ask.
|
|
4
|
+
|
|
5
|
+
## Prohibited
|
|
6
|
+
- Never write new logic without first checking if it already exists in the codebase
|
|
7
|
+
- Never change public interfaces without explicit discussion
|
|
8
|
+
|
|
9
|
+
## Architecture
|
|
10
|
+
- **Closure factories instead of classes.** `function createX(deps) { ... return { ... } }`.
|
|
11
|
+
Keep state in the closure; avoid classes and `this`.
|
|
12
|
+
- **DI at the boundary, closures inside.** A *reusable / exported* factory takes its
|
|
13
|
+
dependencies as an explicit `deps` object, so it can move to its own file unchanged.
|
|
14
|
+
*Inside* a factory, nesting is normal and encouraged — inner factories and helper
|
|
15
|
+
functions freely close over the enclosing scope (that captured state is the whole point
|
|
16
|
+
of closures). Factories don't have to be flat.
|
|
17
|
+
- A factory's returned object is a namespace of functions (functional style,
|
|
18
|
+
an "object with meaning"). Split the return **by audience** when there's more than one —
|
|
19
|
+
`control` (what goes *into* the owning unit) vs `api` (what's exposed *outward*). The
|
|
20
|
+
number of facades follows the need: **one** flat object when there's a single audience,
|
|
21
|
+
**two** (`control`/`api`) for the common in/out split, **more** when distinct consumers
|
|
22
|
+
warrant it (e.g. `control` / `api` / `debug`). Don't force two if one is clearer.
|
|
23
|
+
- **Layering — separate resource / utility / business logic.** Keep the *resource* part
|
|
24
|
+
(sockets, adapters, connections, stores) apart from *utilities* (pure, reusable helpers)
|
|
25
|
+
and from *business logic*. Business logic is **local**: each layer has its own — the
|
|
26
|
+
API-description layer carries the business rules for that API; a higher layer carries the
|
|
27
|
+
business rules of *using* those APIs. Wherever it lives, mark it clearly (section dividers).
|
|
28
|
+
Utility-leaning functions are better *extracted* (own function / file) so they don't blur
|
|
29
|
+
into business logic.
|
|
30
|
+
- **Multi-level facades are fine** — `listen` (or other callbacks) may be installed at
|
|
31
|
+
any nesting depth. Closures are fine. Inner factories and captured state are encouraged.
|
|
32
|
+
- **Expose callbacks outward.** `listen` streams that callers will subscribe to belong in
|
|
33
|
+
the `api` surface — prefer over-exposing event streams to hiding them.
|
|
34
|
+
- Registries/tables — `as const` + inferred literal types (type-safe).
|
|
35
|
+
|
|
36
|
+
## Syntax
|
|
37
|
+
- **`==` / `!=`**, not `===` / `!==` (unless strict comparison is genuinely needed).
|
|
38
|
+
- **Don't annotate function return types** (`: Promise<void>`, `: number`, etc.) — let them
|
|
39
|
+
be inferred. Exception: when the type genuinely helps the reader or narrows inference.
|
|
40
|
+
- Single quotes, 4-space indent, no semicolons.
|
|
41
|
+
- **Named functions over anonymous arrows for anything non-trivial.** A named `function foo()`
|
|
42
|
+
shows up by name in stack traces / logs; an anonymous arrow is `<anonymous>`. So: real logic,
|
|
43
|
+
anything that can throw, async handlers, event callbacks → name them. Trivial one-liners where
|
|
44
|
+
an error is impossible (simple getters, mappers, predicates) — arrow is fine, no need to fuss.
|
|
45
|
+
|
|
46
|
+
## Types
|
|
47
|
+
- Union and primitive aliases — `t` prefix: `tNum`, `tSide`, `tOrderId`.
|
|
48
|
+
- Generic parameters — uppercase `T`, `K`, `Cb`.
|
|
49
|
+
- Derive public types from the implementation: `type X = ReturnType<typeof createX>`.
|
|
50
|
+
|
|
51
|
+
## Comments
|
|
52
|
+
- Section dividers like `// ===...===` with a block heading.
|
|
53
|
+
- Explain "why", not "what". Keep them short.
|
|
54
|
+
## Documentation and release notes
|
|
55
|
+
|
|
56
|
+
- `README.md` is navigation only. Do not put API guides, examples, or long explanations there.
|
|
57
|
+
- The brief public surface lives in `doc/wenay-common2.md`.
|
|
58
|
+
- The extended/rare public surface lives in `doc/wenay-common2-rare.md`.
|
|
59
|
+
- Naming migrations live in `doc/NAMING_RENAMES.md`.
|
|
60
|
+
- Recent changes live in `doc/changes/` as one markdown file per published version, named `<version>.md`.
|
|
61
|
+
- Every release/change intended for publication must add or update the current version file in `doc/changes/` with a short commit-style summary of what changed.
|
|
62
|
+
- Keep only the latest 10 version files in `doc/changes/`; delete older version files when adding a new one.
|
|
63
|
+
- Do not publish until brief docs, rare docs, and the current version change file match the code being published.
|
package/README.md
CHANGED
|
@@ -1,325 +1,9 @@
|
|
|
1
1
|
# wenay-common2
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
## Documentation
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
-
|
|
9
|
-
-
|
|
10
|
-
- **`Decorator.ts`** — function decorators/transformers (before/after call, result processing)
|
|
11
|
-
- **`MemoFunc.ts`** — memoization with TTL and limits, LRU eviction
|
|
12
|
-
|
|
13
|
-
### 📊 **data/** — Data Structures and Serialization
|
|
14
|
-
- **`List.ts`** — doubly linked list `CList` with immutable interfaces
|
|
15
|
-
- **`ListNodeAnd.ts`** — alternative doubly linked list nodes with a sentinel element
|
|
16
|
-
- **`ByteStream.ts`** — binary serialization: `ByteStreamW` (write) and `ByteStreamR` (read), support for numeric types and arrays
|
|
17
|
-
- **`objectPath.ts`** — working with object paths (`objectGetValueByPath`, `objectSetValueByPath`, iteration via `iterateDeepObjectEntries`)
|
|
18
|
-
|
|
19
|
-
### ⏱️ **async/** — Async Utilities
|
|
20
|
-
- **`waitRun.ts`** — throttle/debounce (`enhancedWaitRun`), async queues (`createAsyncQueue`, `queueRun`), task queue with readiness control (`createTaskQueue`)
|
|
21
|
-
- **`promiseProgress.ts`** — processing an array of promises with success/error subscriptions
|
|
22
|
-
- **`createIterableObject.ts`** — proxy for iterable objects (readonly/read-write)
|
|
23
|
-
|
|
24
|
-
### 🔔 **events/** — Events and Subscriptions
|
|
25
|
-
- **`Listen.ts`** — subscription system (`listen`, `createListen`), `isListenCallback` check
|
|
26
|
-
- **`event.ts`** — event handler collections (`CObjectEventsArr`, `CObjectEventsList`)
|
|
27
|
-
- **`SocketBuffer.ts`** — socket callback buffering (`socketBuffer`, `listenSnapshot`)
|
|
28
|
-
- **`SocketServerHook.ts`** — tag-based subscription wrappers for socket communication (`SocketServerHook`, `WebSocketServerHook`)
|
|
29
|
-
- **`joinListens.ts`** — merging multiple subscription streams into one (`joinListens`)
|
|
30
|
-
- **`listen-socket.ts`** — bridge between event system and RPC (`listenSocket`, `listenSocketFirst`, `listenSocketAll`, `listenSocketSmart`)
|
|
31
|
-
|
|
32
|
-
### 🌐 **rpc/** — RPC System
|
|
33
|
-
**Core:**
|
|
34
|
-
- **`rpc-protocol.ts`** — RPC protocol (packet types `Pkt`, `SocketTmpl`)
|
|
35
|
-
- **`rpc-client.ts`** — RPC client (`createRpcClient`, `ClientAPI`)
|
|
36
|
-
- **`rpc-server.ts`** — RPC server (`createRpcServer`, hooks `PromiseServerHooks`)
|
|
37
|
-
- **`rpc-walk.ts`** — structure traversal (callback serialization, `pack`/`unpack`, `resolveCA`)
|
|
38
|
-
- **`rpc-limits.ts`** — security limits (`RpcLimits`, `PayloadLimitError`, `isSafeKey`)
|
|
39
|
-
- **`rpc-dynamic.ts`** — dynamic object marking (`noStrict`, `isNoStrict`)
|
|
40
|
-
- **`id-pool.ts`** — reusable ID pool (`createIdPool`)
|
|
41
|
-
|
|
42
|
-
**Automation:**
|
|
43
|
-
- **`rpc-client-auto.ts`** — client with automatic event subscription (`createRpcClientAuto`, modes `smart`/`first`/`all`)
|
|
44
|
-
- **`rpc-server-auto.ts`** — server with automatic subscription handling (`createRpcServerAuto`)
|
|
45
|
-
- **`listen-deep.ts`** — deep object transformation for subscriptions (`deepListenFirst`, `deepListenAll`, `deepListenSmart`)
|
|
46
|
-
|
|
47
|
-
### 📡 **network/** — Network Utilities
|
|
48
|
-
- **`WebHook3.ts`** — webhook server/client on Express
|
|
49
|
-
- **Server:** `createWebhookServer({ authToken, port, app? })`
|
|
50
|
-
- **Client:** `createWebhookClient({ serverUrl, clientPort, authToken, autoRenew? })`
|
|
51
|
-
- Support for legacy (`tag`) and new (`tags`) format in `subscribers.json`
|
|
52
|
-
|
|
53
|
-
### ⏰ **time/** — Time Handling
|
|
54
|
-
- **`Time.ts`** — timeframes (`TF`), periodics (`Period`), time formatting, date conversion (`timeToStr_*`, `timeLocalToStr_*`)
|
|
55
|
-
- **`funcTimeWait.ts`** — request time tracking by keys (`funcTimeW`, `FuncTimeWait`)
|
|
56
|
-
|
|
57
|
-
### 🎨 **utils/** — Helper Utilities
|
|
58
|
-
- **`Color.ts`** — color handling (`ColorString`, `rgb`, generators `colorGenerator`, similarity check `isSimilarColors`)
|
|
59
|
-
- **`Math.ts`** — Pearson correlation calculation (`CorrelationRollingByBuffer`)
|
|
60
|
-
- **`isProxy.ts`** — Proxy object detection (Node.js + browser)
|
|
61
|
-
- **`fsKeyVolume.ts`** — file system key-value storage (`saveKeyValue`)
|
|
62
|
-
- **`inputAutoStep.ts`** — automatic step control for `<input>` (`SetAutoStepForElement`)
|
|
63
|
-
- **`node_console.ts`** — enhanced console output with source maps (clickable links in IDE)
|
|
64
|
-
|
|
65
|
-
---
|
|
66
|
-
|
|
67
|
-
## 🚀 Quick Start
|
|
68
|
-
|
|
69
|
-
### Installation
|
|
70
|
-
```bash
|
|
71
|
-
npm install wenay-common2
|
|
72
|
-
```
|
|
73
|
-
|
|
74
|
-
Here is a comprehensive guide in a concise style. I've taken into account the architecture, backend, frontend (with the new `Hub` pattern), serialization nuances, limits, and hooks.
|
|
75
|
-
|
|
76
|
-
# wenay-common2 RPC: Complete Guide
|
|
77
|
-
|
|
78
|
-
Bidirectional, strongly-typed RPC protocol over sockets (Socket.IO or similar).
|
|
79
|
-
**Essence:** Server exposes a nested JS object $\to$ Client receives a typed proxy.
|
|
80
|
-
|
|
81
|
-
---
|
|
82
|
-
|
|
83
|
-
## 1. Architecture and Limitations
|
|
84
|
-
|
|
85
|
-
* **Multiplexing:** A single physical socket hosts independent channels (`socketKey`), each with its own API object.
|
|
86
|
-
* **Data Types:** Works with JSON-compatible data plus `Date`, `Map`, `Set`, `RegExp`, and `BigInt`. Class instances are sent as plain enumerable object data; methods/prototypes are not preserved.
|
|
87
|
-
* **Security (RpcLimits):** Server is protected from DDoS attacks. Strict limits on: `maxDepth`, `maxKeys`, `maxArrayLen`, `maxStringLen`, `maxCallbacks`. Exceeding throws `PayloadLimitError`.
|
|
88
|
-
|
|
89
|
-
---
|
|
90
|
-
|
|
91
|
-
## 2. Server (Backend)
|
|
92
|
-
|
|
93
|
-
### 2.1 Socket Connection
|
|
94
|
-
```typescript
|
|
95
|
-
import { createRpcServerAuto, listen } from "wenay-common2";
|
|
96
|
-
|
|
97
|
-
io.sockets.on('connection', (socket) => {
|
|
98
|
-
// 1. Create unsubscribe trigger for memory cleanup
|
|
99
|
-
const [stop, listenStop] = listen<[]>();
|
|
100
|
-
socket.on('disconnect', stop);
|
|
101
|
-
|
|
102
|
-
// 2. Initialize RPC channel on this socket
|
|
103
|
-
createRpcServerAuto({
|
|
104
|
-
socket,
|
|
105
|
-
socketKey: "mainAPI", // Channel identifier
|
|
106
|
-
object: buildFacade(client), // Target API object
|
|
107
|
-
disconnectListen: listenStop, // Auto-unsubscribe from Listen on disconnect
|
|
108
|
-
debug: process.env.DEV, // Packet logging
|
|
109
|
-
});
|
|
110
|
-
});
|
|
111
|
-
```
|
|
112
|
-
|
|
113
|
-
### 2.2 Building API Object (Facade)
|
|
114
|
-
The object is traversed by the server to build a "Schema" that is sent to the client.
|
|
115
|
-
```typescript
|
|
116
|
-
import { noStrict, listen } from "wenay-common2";
|
|
117
|
-
|
|
118
|
-
// Create pub/sub event system
|
|
119
|
-
const [sendEvent, listenEvent] = listen<[string]>();
|
|
120
|
-
|
|
121
|
-
export function buildFacade(client) {
|
|
122
|
-
const role = (...roles) => hasRole(client, roles) ? true : null;
|
|
123
|
-
|
|
124
|
-
return {
|
|
125
|
-
// 1. Regular method
|
|
126
|
-
ping: () => "pong",
|
|
127
|
-
|
|
128
|
-
// 2. Nested namespaces + Role model
|
|
129
|
-
// If role() returns null, the method won't be sent to the client (returns null in schema)
|
|
130
|
-
admin: {
|
|
131
|
-
deleteUser: role("admin") && ((id) => db.delete(id)),
|
|
132
|
-
},
|
|
133
|
-
|
|
134
|
-
// 3. Dynamic objects (Proxy, ORM)
|
|
135
|
-
// Wrap in noStrict so the server doesn't try to read keys.
|
|
136
|
-
// Client will work with it in "blind" mode (without schema).
|
|
137
|
-
dbRef: noStrict(getProxyDb()),
|
|
138
|
-
|
|
139
|
-
// 4. Events
|
|
140
|
-
// Client will receive a Listen surface: .on(cb), .once(cb), .close()
|
|
141
|
-
events: { listenEvent },
|
|
142
|
-
|
|
143
|
-
// 5. Method with callback in arguments
|
|
144
|
-
// Callback lives ONLY while await is executing! After return, client deletes it.
|
|
145
|
-
stream: async (cb: (chunk: number) => void) => {
|
|
146
|
-
for(let i=0; i<10; i++) { cb(i); await sleep(50); }
|
|
147
|
-
return "done";
|
|
148
|
-
}
|
|
149
|
-
};
|
|
150
|
-
}
|
|
151
|
-
```
|
|
152
|
-
|
|
153
|
-
### 2.3 Server Hooks (Interceptors)
|
|
154
|
-
Use hooks to validate incoming packets.
|
|
155
|
-
```typescript
|
|
156
|
-
createRpcServerAuto({
|
|
157
|
-
/*...*/
|
|
158
|
-
hooks: {
|
|
159
|
-
onRequest: async ({ key, request, fnName, fn }) => {
|
|
160
|
-
// Return false to block the call
|
|
161
|
-
return true;
|
|
162
|
-
},
|
|
163
|
-
onInvalid: ({ reason, key, request }) => {
|
|
164
|
-
console.warn(`RPC Attack/Error [${reason}]:`, key);
|
|
165
|
-
}
|
|
166
|
-
}
|
|
167
|
-
})
|
|
168
|
-
```
|
|
169
|
-
|
|
170
|
-
---
|
|
171
|
-
|
|
172
|
-
## 3. Client (Frontend)
|
|
173
|
-
|
|
174
|
-
**Hub Pattern:** The frontend library doesn't depend on `socket.io-client`. Developer injects the socket factory into `createRpcClientHub`.
|
|
175
|
-
|
|
176
|
-
### 3.1 Hub Initialization
|
|
177
|
-
```typescript
|
|
178
|
-
import { io } from "socket.io-client";
|
|
179
|
-
import { createRpcClientHub, rpc } from "wenay-common2";
|
|
180
|
-
import type { MainFacade } from "../server/facade";
|
|
181
|
-
|
|
182
|
-
export const Api = createRpcClientHub(
|
|
183
|
-
// 1. Socket factory (DI)
|
|
184
|
-
(token) => io("http://localhost:4021", {
|
|
185
|
-
transports: ["websocket"],
|
|
186
|
-
query: token ? { token } : {}
|
|
187
|
-
}),
|
|
188
|
-
|
|
189
|
-
// 2. Channel registration
|
|
190
|
-
// rpc() accepts Facade type. Property name ("mainAPI") becomes socketKey.
|
|
191
|
-
(rpc) => ({
|
|
192
|
-
mainAPI: rpc<MainFacade>(),
|
|
193
|
-
})
|
|
194
|
-
);
|
|
195
|
-
```
|
|
196
|
-
|
|
197
|
-
### 3.2 Connection Lifecycle
|
|
198
|
-
```typescript
|
|
199
|
-
// Listen to statuses
|
|
200
|
-
Api.onConnect((count) => console.log(`Socket connected (attempt ${count})`));
|
|
201
|
-
|
|
202
|
-
// Initiate connect. Creates socket, all channels automatically start.
|
|
203
|
-
await Api.connect("USER_SECRET_TOKEN");
|
|
204
|
-
|
|
205
|
-
// Disconnect
|
|
206
|
-
// await Api.connect(null);
|
|
207
|
-
```
|
|
208
|
-
|
|
209
|
-
### 3.3 Call Modes
|
|
210
|
-
Access API channel: `Api.facade.mainAPI`. Hub initializes all channels, so schema loads automatically.
|
|
211
|
-
|
|
212
|
-
```typescript
|
|
213
|
-
const api = Api.facade.mainAPI;
|
|
214
|
-
|
|
215
|
-
// Wait for schema (REQUIRED before UI render)
|
|
216
|
-
await api.ready();
|
|
217
|
-
|
|
218
|
-
// --- 1. STRICT (Recommended) ---
|
|
219
|
-
// Safe call. Use `?.` since method may be `null` (closed by roles).
|
|
220
|
-
// If method is closed, returns `undefined` without sending network request.
|
|
221
|
-
const res = await api.strict.admin?.deleteUser?.(5);
|
|
222
|
-
|
|
223
|
-
// --- 2. FUNC (Standard) ---
|
|
224
|
-
const res2 = await api.func.ping();
|
|
225
|
-
|
|
226
|
-
// --- 3. PIPE (Pipeline) ---
|
|
227
|
-
// Entire chain goes to server in ONE network packet.
|
|
228
|
-
const data = await api.pipe.dbRef.users.find(1).getName();
|
|
229
|
-
|
|
230
|
-
// --- 4. SPACE (Fire-and-Forget) ---
|
|
231
|
-
// Doesn't wait for response, Promise resolves immediately.
|
|
232
|
-
api.space.admin.logAction("clicked");
|
|
233
|
-
```
|
|
234
|
-
|
|
235
|
-
### 3.4 Client Subscriptions (Listen)
|
|
236
|
-
`createRpcServerAuto` exposes server `listen` values as RPC Listen nodes. New code uses `on`/`once` and keeps the returned `off` handle. For TypeScript, project `client.func` to `DeepSocketListen<ServerFacade>`; this mirrors the runtime shape and keeps event argument types.
|
|
237
|
-
```typescript
|
|
238
|
-
import type { DeepSocketListen } from "wenay-common2";
|
|
239
|
-
|
|
240
|
-
function webListen<T extends object>(client: { func: unknown }) {
|
|
241
|
-
return client.func as DeepSocketListen<T>;
|
|
242
|
-
}
|
|
243
|
-
|
|
244
|
-
const events = webListen<MainFacade>(api).events;
|
|
245
|
-
|
|
246
|
-
// Subscribe
|
|
247
|
-
const off = events.listenEvent.on((msg) => {
|
|
248
|
-
console.log("Push from server:", msg);
|
|
249
|
-
});
|
|
250
|
-
|
|
251
|
-
// Unsubscribe. The handle is callable and also thenable.
|
|
252
|
-
off();
|
|
253
|
-
// await off; // waits until the stream ends
|
|
254
|
-
|
|
255
|
-
// One event, then automatic unsubscribe
|
|
256
|
-
const done = events.listenEvent.once((msg) => {
|
|
257
|
-
console.log("First push:", msg);
|
|
258
|
-
});
|
|
259
|
-
await done;
|
|
260
|
-
```
|
|
261
|
-
|
|
262
|
-
Compatibility names `.callback(cb)`, `.removeCallback()`, and `.unsubscribe()` still exist for old clients, but they are not the recommended API.
|
|
263
|
-
|
|
264
|
-
|
|
265
|
-
### 3.5 Request Management and Debug
|
|
266
|
-
Each facade has a system object `api` for low-level control, as well as a couple of methods in the root:
|
|
267
|
-
|
|
268
|
-
```typescript
|
|
269
|
-
const { api, abortAll, schema } = Api.facade.mainAPI;
|
|
270
|
-
|
|
271
|
-
// --- 1. Monitoring and Debug ---
|
|
272
|
-
api.pending(); // Current number of pending responses (Promises)
|
|
273
|
-
api.callbacks(); // Current number of live callback ids in memory
|
|
274
|
-
api.log(true); // Enable logging of all incoming/outgoing packets to console
|
|
275
|
-
|
|
276
|
-
// --- 2. Targeted cleanup (inside .api) ---
|
|
277
|
-
api.clearPromises(true); // Reject (cancel) all current requests
|
|
278
|
-
api.clearCallbacks(); // Force clear all callbacks
|
|
279
|
-
api.remove(myFunc); // Force-release a specific callback id (alias: .end)
|
|
280
|
-
|
|
281
|
-
// --- 3. Global facade methods ---
|
|
282
|
-
abortAll("User logout"); // Hard reset: reject all promises with RPC_ABORT error + clear all callback ids
|
|
283
|
-
const map = schema(); // Get raw schema tree (MAP) sent by server
|
|
284
|
-
```
|
|
285
|
-
|
|
286
|
-
|
|
287
|
-
---
|
|
288
|
-
|
|
289
|
-
## 4. Advanced Features
|
|
290
|
-
|
|
291
|
-
### 4.1 Listen Argument Interception Modes
|
|
292
|
-
When using events, the client auto-handler (`mode: "smart"`) by default flexibly adapts arguments. If server sends one argument — it comes as a value, if multiple — as an array.
|
|
293
|
-
If you create a client manually without Hub, you can set a strict mode:
|
|
294
|
-
```typescript
|
|
295
|
-
// "first" — listener always receives only the first argument
|
|
296
|
-
// "all" — listener always receives all arguments
|
|
297
|
-
// "smart" — (default) auto-detection
|
|
298
|
-
const autoApi = createRpcClientAuto(api.func, { mode: "first" });
|
|
299
|
-
```
|
|
300
|
-
|
|
301
|
-
|
|
302
|
-
### 4.2 Manual Callback Termination from Server
|
|
303
|
-
This is a low-level escape hatch for callbacks passed as ordinary function arguments. For Listen subscriptions prefer `off()`/`.once()`.
|
|
304
|
-
```typescript
|
|
305
|
-
import { endCallback } from "wenay-common2";
|
|
306
|
-
|
|
307
|
-
async function myMethod(cb: (data: any) => void) {
|
|
308
|
-
cb("chunk 1");
|
|
309
|
-
endCallback(cb); // Alias: rpcEndCallback. Sends "___STOP" to client.
|
|
310
|
-
// Client callback id is deleted, subsequent cb() calls won't go anywhere.
|
|
311
|
-
}
|
|
312
|
-
```
|
|
313
|
-
|
|
314
|
-
### 4.3 Call / Apply Support on Client
|
|
315
|
-
The client proxy can transparently handle standard JS `call` and `apply` calls. Server correctly normalizes them ("collapses" the path):
|
|
316
|
-
```typescript
|
|
317
|
-
// Both variants correctly call `api.users.create("Ivan")` on server
|
|
318
|
-
await api.func.users.create.call(null, "Ivan");
|
|
319
|
-
await api.func.users.create.apply(null, ["Ivan"]);
|
|
320
|
-
```
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
### 4.4 Transparent PIPE Request Transit
|
|
324
|
-
For microservice architecture. If your Node server itself is a client of another RPC node (via `pipe`), it can "pass through" the remainder of the pipe chain further using `__executeRemainingPipe`, without waiting for an intermediate response.
|
|
325
|
-
This covers the important RPC mechanics (Listen `on/once/off`, `endCallback`, `call/apply`, `pipe-transit`) while maintaining readability.
|
|
5
|
+
- Brief API cheat sheet: [`doc/wenay-common2.md`](doc/wenay-common2.md)
|
|
6
|
+
- Extended API cheat sheet: [`doc/wenay-common2-rare.md`](doc/wenay-common2-rare.md)
|
|
7
|
+
- Naming migrations: [`doc/NAMING_RENAMES.md`](doc/NAMING_RENAMES.md)
|
|
8
|
+
- Recent changes: [`doc/changes/`](doc/changes/)
|
|
9
|
+
- Project rules for AI/code maintenance: [`CLAUDE.md`](CLAUDE.md)
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
# Карта переименований
|
|
2
|
+
|
|
3
|
+
Breaking migration: старые имена не оставляем алиасами.
|
|
4
|
+
|
|
5
|
+
| Было | Стало |
|
|
6
|
+
| --- | --- |
|
|
7
|
+
| `UseListen` | `listen` |
|
|
8
|
+
| `UseListenStore` | `listenStore` |
|
|
9
|
+
| `UseListen2` | `slimListen` |
|
|
10
|
+
| `toListen2` | `toSlimListen` |
|
|
11
|
+
| `UseListenTransform` | `mapListen` |
|
|
12
|
+
| `funcListenCallbackBase` | `createListen` |
|
|
13
|
+
| `funcListenCallbackFast` | `createFastListen` |
|
|
14
|
+
| `funcListenCallbackStore` | `createStoreListen` |
|
|
15
|
+
| `funcListenCore` | `createListenCore` |
|
|
16
|
+
| `addListen` | `on` |
|
|
17
|
+
| `removeListen` | `off` |
|
|
18
|
+
| `eventClose` | `onClose` |
|
|
19
|
+
| `removeEventClose(cb)` | `const off = onClose(cb); off()` |
|
|
20
|
+
| `addListenClose` | `closeOn` |
|
|
21
|
+
| `tSubHandle` | `SubscriptionHandle` |
|
|
22
|
+
| `PromiseArrayListen` | `promiseProgress` |
|
|
23
|
+
| `listenOk` / `listenError` | `onOk` / `onError` |
|
|
24
|
+
| `promise.all()` / `promise.allSettled()` | `all()` / `allSettled()` |
|
|
25
|
+
| `getData()` / `status()` | `items()` / `stats()` |
|
|
26
|
+
| `realSocket2` | `SocketSource` |
|
|
27
|
+
| `getTypeCallback` | `SocketPayload` |
|
|
28
|
+
| `socketBuffer3` | `socketBuffer` |
|
|
29
|
+
| `funcListenCallbackSnapshot` | `listenSnapshot` |
|
|
30
|
+
| `createRpcServerAuto2` | `createRpcServerAutoDetect` |
|
|
31
|
+
| `UseReplayListen` | `replayListen` |
|
|
32
|
+
| `ListenNext` | root exports from `wenay-common2` (`listen`, `listenStore`, `slimListen`) |
|
|
33
|
+
| `wenay-common2/listen2` | removed; use root exports from `wenay-common2` |
|
|
34
|
+
| `ObserveAll2` | `Observe` |
|
|
35
|
+
| `wenay-common2/observe-all2` | `wenay-common2/observe` |
|
|
36
|
+
| `src/Common/ObserveAll2` | `src/Common/Observe` |
|
|
@@ -0,0 +1,56 @@
|
|
|
1
|
+
# Recommendations — Listen / Observe / RPC integration
|
|
2
|
+
|
|
3
|
+
Status refreshed 2026-07-08. This document is now a current checklist, not the
|
|
4
|
+
old 2026-07-02 migration scratchpad. Public code should import the canonical
|
|
5
|
+
surfaces (`wenay-common2`, `wenay-common2/observe`, `wenay-common2/replay`, or
|
|
6
|
+
`src/Common/events/Listen` inside the repo). Numbered implementation artifacts
|
|
7
|
+
such as `Listen3` and `reactive2` are compatibility shims only.
|
|
8
|
+
|
|
9
|
+
## 1. Naming cleanup
|
|
10
|
+
|
|
11
|
+
- **DONE** `src/Common/events/Listen.ts` is the real implementation file.
|
|
12
|
+
`Listen3.ts` remains only as an internal deep-import shim.
|
|
13
|
+
- **DONE** `src/Common/Observe/reactive.ts` is the real reactive implementation.
|
|
14
|
+
`reactive2.ts` remains only as an internal deep-import shim.
|
|
15
|
+
- **DONE** the oracle folder is `observe/`, not `observable2/`.
|
|
16
|
+
- **RULE** new code and docs should not introduce `UseListen`, `Listen3`,
|
|
17
|
+
`reactive2`, or `observable2` outside `NAMING_RENAMES.md`, historical `doc/changes/` notes, or shim comments.
|
|
18
|
+
|
|
19
|
+
## 2. Observe store status
|
|
20
|
+
|
|
21
|
+
- **DONE** circular `Map`/`Set` snapshots are guarded by `seen` before container
|
|
22
|
+
traversal.
|
|
23
|
+
- **DONE** `createStoreMirror.sync` reports failed re-pulls through
|
|
24
|
+
`StoreSyncOpts.onError`; initial pull failure still rejects the awaited sync.
|
|
25
|
+
- **DONE** path cache identity separates dotted keys and distinct symbols.
|
|
26
|
+
- **DONE** brief/rare docs describe the store layer as separate from the coarse
|
|
27
|
+
reactive core.
|
|
28
|
+
- **OPEN** `_nodeCache` and `_counts` are unbounded for dynamic path spaces.
|
|
29
|
+
Either document bounded-path expectations for applications or add eviction.
|
|
30
|
+
- **OPEN** `schedule` / `createDrained` in `store.ts` still duplicate scheduler
|
|
31
|
+
logic from `reactive.ts`; extract only if more local schedulers appear.
|
|
32
|
+
|
|
33
|
+
## 3. Store manager direction
|
|
34
|
+
|
|
35
|
+
The universal manager should stay above `Observe.createStore` / replay / offline.
|
|
36
|
+
It should not make the store core know about products, routes, user habits, or
|
|
37
|
+
large-resource policy. The manager owns resource declarations, priority, tags,
|
|
38
|
+
usage scoring, explicit-only gates, and start/stop lifecycle.
|
|
39
|
+
|
|
40
|
+
Current implementation target: `Observe.createStoreManager` with `managedStore`
|
|
41
|
+
builders for `mirror`, `replay`, and `offline` resources.
|
|
42
|
+
|
|
43
|
+
## 4. RPC current over wire
|
|
44
|
+
|
|
45
|
+
Still open. The client dedup key already includes non-function args, but the
|
|
46
|
+
server subscription path does not yet whitelist and forward `{current:true}` to
|
|
47
|
+
Listen store decorators. Do not document `{current:true}` over RPC as supported
|
|
48
|
+
until server forwarding and late-join replay are implemented together.
|
|
49
|
+
|
|
50
|
+
## 5. Suggested order
|
|
51
|
+
|
|
52
|
+
1. Keep naming cleanup green with `rg` before release.
|
|
53
|
+
2. Add tests for `createStoreManager` planning, explicit gates, mirror sync, and
|
|
54
|
+
offline/replay startup.
|
|
55
|
+
3. Update brief and rare docs only after tests pass.
|
|
56
|
+
4. Bump patch version, build, then publish from `dist`.
|
|
@@ -0,0 +1,81 @@
|
|
|
1
|
+
# Snapshot + Sequenced Delta Line — universal state sync
|
|
2
|
+
|
|
3
|
+
One pattern for everything: **keyframe (snapshot) + numbered line of deltas + recovery via fresh keyframe**.
|
|
4
|
+
Payload-agnostic: store patches, ticks, any events. Same figure as market-data feeds
|
|
5
|
+
(incremental + snapshot channel), DB replication (basebackup + WAL), video (I/P frames), Kafka.
|
|
6
|
+
|
|
7
|
+
## Invariants
|
|
8
|
+
|
|
9
|
+
- **seq is the coordinate, time is an attribute.** Every journaled event: `{seq, ts, event}`.
|
|
10
|
+
seq = monotonic counter → strict order, gap detection, exact replay→live handover.
|
|
11
|
+
Time is for humans only: "rewind to 12:00" resolves to nearest keyframe ≤ ts → its seq.
|
|
12
|
+
- **Keyframe = state as of seq S.** Rule of the whole system:
|
|
13
|
+
`keyframe(S) + events S+1…K = exact state at K`.
|
|
14
|
+
- **Memory stays external** (lambda providers), decorators own no data they don't have to.
|
|
15
|
+
- **Additive only.** The canonical Listen surface is untouched; new surface = decorator + opts.
|
|
16
|
+
- The live store is never reset: archiving = take a labeled copy, keep mutating the same store.
|
|
17
|
+
|
|
18
|
+
## The killer property (drives everything)
|
|
19
|
+
|
|
20
|
+
A lagging/late/reconnecting consumer never gets a queue backlog — it gets a **fresh keyframe
|
|
21
|
+
+ the line from there**. This is both the catch-up mechanism AND the backpressure policy
|
|
22
|
+
(conflation + snapshot recovery). No unbounded buffering, ever.
|
|
23
|
+
|
|
24
|
+
## Layers
|
|
25
|
+
|
|
26
|
+
### A. Listen replay decorator — `withReplayListen` (universal, payload-agnostic)
|
|
27
|
+
|
|
28
|
+
Decorator over `ListenApi`, exact shape of `withStoreListen`:
|
|
29
|
+
|
|
30
|
+
```
|
|
31
|
+
withReplayListen(base, {
|
|
32
|
+
current, // () => keyframe — external memory lambda (exists: layer 3)
|
|
33
|
+
history?: number, // ring buffer size N (internal) …
|
|
34
|
+
getSince?: (seq) => events[], // …or external journal lambda (memory outside — preferred)
|
|
35
|
+
})
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
Subscription modes:
|
|
39
|
+
- `on(cb)` — live only (unchanged)
|
|
40
|
+
- `on(cb, {current: true})` — keyframe + live (unchanged, layer 3)
|
|
41
|
+
- `on(cb, {since: seq})` — catch-up: replay journal from seq+1, queue live events by seq
|
|
42
|
+
during replay, dedup by seq, seamless switch to live. seq evicted from journal →
|
|
43
|
+
fallback: keyframe + live.
|
|
44
|
+
|
|
45
|
+
The handover (replay→live without gap or dup) is the ONLY subtle code in the plan.
|
|
46
|
+
Everything else is mechanics.
|
|
47
|
+
|
|
48
|
+
### B. Observe store integration
|
|
49
|
+
|
|
50
|
+
- `exposeStore`: patch journal — every emitted `StorePatch` gets seq; keep last N (ring).
|
|
51
|
+
`snapshot()` responses carry the seq they correspond to.
|
|
52
|
+
- `createStoreMirror`: sync = "I have seq K" → tail of patches, or snapshot(S) + live from S+1.
|
|
53
|
+
Reconnect stops costing a full snapshot when the tail suffices.
|
|
54
|
+
- Uses layer A; store adds nothing but "patch as the event type".
|
|
55
|
+
|
|
56
|
+
### C. History storage (optional, lazy, fully external)
|
|
57
|
+
|
|
58
|
+
For seek into the past — not needed for live sync:
|
|
59
|
+
- periodic archiving: keyframe every N events OR T seconds, whichever first (GOP tradeoff:
|
|
60
|
+
dense = fast seek / more space, sparse = cheap / longer replay);
|
|
61
|
+
- providers: `getKeyframe(seqOrTs)`, `getEvents(from, to)` — file/DB/anything, behind lambdas;
|
|
62
|
+
- a history reader is the SAME subscriber interface: subscribe `{since}` → archive replay →
|
|
63
|
+
journal catch-up → live. One mechanism for playback and live.
|
|
64
|
+
|
|
65
|
+
### D. Transport hardening (independent add-ons, any order, later)
|
|
66
|
+
|
|
67
|
+
- **Per-client conflation**: outgoing buffer over threshold → stop deltas for that client,
|
|
68
|
+
mark "needs keyframe"; buffer drained → fresh snapshot + resume from current seq.
|
|
69
|
+
- **Serialize once, fan out**: pack a tick once, send bytes to all subscribers
|
|
70
|
+
(extends existing Caps.COMPACT / shape packing).
|
|
71
|
+
- **Coalescing**: N updates of one key while client lagged → send only the last.
|
|
72
|
+
- **Binary passthrough** in `rpc-walk`: TypedArray/ArrayBuffer/Buffer as leaf values
|
|
73
|
+
(socket.io carries binary natively; today walk() mangles them into `{0:…,1:…}`).
|
|
74
|
+
|
|
75
|
+
## Stages
|
|
76
|
+
|
|
77
|
+
1. `withReplayListen` decorator + seq handover (layer A) — the core, small.
|
|
78
|
+
2. Patch journal in `exposeStore` + `since`-sync in mirror (layer B).
|
|
79
|
+
3. Conflation + snapshot recovery on the wire (first item of D) — this is what turns
|
|
80
|
+
"semi-pro" into pro for fan-out.
|
|
81
|
+
4. C and the rest of D — on demand.
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
# 1.0.63
|
|
2
|
+
|
|
3
|
+
- Add `Observe.createStoreManager` and `Observe.managedStore` resource builders for declarative mirror/replay/offline store startup.
|
|
4
|
+
- Add manager planning with priority, tags, usage scoring, `large`, and `explicitOnly` gates.
|
|
5
|
+
- Add manager lifecycle helpers: `start`, `startPlanned`, `stop`, `stopAll`, `get`, `touch`, `usage`, `statusListen`, and typed `handles`.
|
|
6
|
+
- Rename public-facing Observe oracle folder from `observable2` to `observe` and make `Listen.ts` / `reactive.ts` the canonical source files, leaving numbered files as compatibility shims.
|
|
7
|
+
- Update brief and extended API docs for the store manager and current naming.
|
|
8
|
+
- Publish package `wenay-common2@1.0.63`.
|
|
@@ -0,0 +1,10 @@
|
|
|
1
|
+
# 1.0.64
|
|
2
|
+
|
|
3
|
+
- Make `README.md` navigation-only and point it to the brief docs, rare docs, naming migrations, recent changes, and project rules.
|
|
4
|
+
- Add the `doc/changes/` release-note catalog rule to `CLAUDE.md`: one file per published version, commit-style summary, latest 10 versions only.
|
|
5
|
+
- Add the recent changes catalog README and keep the current release notes under `doc/changes/`.
|
|
6
|
+
- Package documentation, `CLAUDE.md`, and `rpc.md` into `dist` so README links work in the npm package.
|
|
7
|
+
- Restore the `noStrict` dynamic-map contract in the extended docs after removing a duplicated store-manager block.
|
|
8
|
+
- Move regression oracles to the canonical `observe/reactive` names after the `observable2/reactive2` rename.
|
|
9
|
+
- Normalize npm repository metadata.
|
|
10
|
+
- Publish package `wenay-common2@1.0.64`.
|
|
@@ -0,0 +1,8 @@
|
|
|
1
|
+
# Recent changes
|
|
2
|
+
|
|
3
|
+
This directory keeps one file per published version.
|
|
4
|
+
|
|
5
|
+
Rules:
|
|
6
|
+
- file name: `<version>.md`, for example `1.0.63.md`;
|
|
7
|
+
- each file contains a short commit-style summary of what changed;
|
|
8
|
+
- keep only the latest 10 version files, deleting older version files when adding a new one.
|