@causal-order/dedupe 1.0.1 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +10 -114
- package/package.json +5 -1
package/LICENSE
CHANGED
|
@@ -1,21 +1,21 @@
|
|
|
1
|
-
MIT License
|
|
2
|
-
|
|
3
|
-
Copyright (c) 2026 Gazali Ahmad
|
|
4
|
-
|
|
5
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
-
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
-
in the Software without restriction, including without limitation the rights
|
|
8
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
-
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
-
furnished to do so, subject to the following conditions:
|
|
11
|
-
|
|
12
|
-
The above copyright notice and this permission notice shall be included in all
|
|
13
|
-
copies or substantial portions of the Software.
|
|
14
|
-
|
|
15
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
-
SOFTWARE.
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Gazali Ahmad
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
CHANGED
|
@@ -1,22 +1,18 @@
|
|
|
1
1
|
# @causal-order/dedupe
|
|
2
2
|
|
|
3
|
-
Deduplication support for `causal-order
|
|
3
|
+
Deduplication support for `causal-order` event streams.
|
|
4
4
|
|
|
5
|
-
Version `1.0.
|
|
6
|
-
|
|
7
|
-
Release notes: [CHANGELOG.md](./CHANGELOG.md)
|
|
5
|
+
Version `1.0.2`.
|
|
8
6
|
|
|
9
7
|
Runtime compatibility:
|
|
10
8
|
|
|
11
|
-
- Node `20+`
|
|
12
|
-
- Node `24+` recommended for local development in this repo
|
|
9
|
+
- Node `20+`
|
|
13
10
|
- ESM-only package
|
|
14
11
|
|
|
15
|
-
The repo currently exposes a single gateway class in [src/dedupe.ts](./src/dedupe.ts) and includes runtime scripts in [test/](./test) that simulate multi-process event delivery over localhost TCP.
|
|
16
|
-
|
|
17
12
|
## What It Does
|
|
18
13
|
|
|
19
|
-
`DedupeGateway` keeps a sliding-window cache of event identities and lets you
|
|
14
|
+
`DedupeGateway` keeps a sliding-window cache of event identities and lets you
|
|
15
|
+
drop repeat deliveries before handing events to `causal-order`.
|
|
20
16
|
|
|
21
17
|
An event is deduplicated by:
|
|
22
18
|
|
|
@@ -49,17 +45,17 @@ if (dedupe.filter(event)) {
|
|
|
49
45
|
dedupe.cleanup();
|
|
50
46
|
```
|
|
51
47
|
|
|
52
|
-
For local development in this repo, importing from `./src/dedupe.ts` is still fine when running through the TS toolchain.
|
|
53
|
-
|
|
54
48
|
## API
|
|
55
49
|
|
|
56
|
-
### `new DedupeGateway(
|
|
50
|
+
### `new DedupeGateway(options)`
|
|
57
51
|
|
|
58
|
-
|
|
52
|
+
Creates a dedupe gateway. The `options` object is optional.
|
|
53
|
+
|
|
54
|
+
Options:
|
|
59
55
|
|
|
60
56
|
- `slidingWindowSeconds`: initial lookback window, default `180`
|
|
61
57
|
- `maxSlidingWindowSeconds`: hard upper bound for dynamic window growth, default `300`
|
|
62
|
-
- `nowProvider` or `now_provider`: function
|
|
58
|
+
- `nowProvider` or `now_provider`: function that returns the current time in milliseconds, compatible with `BigInt`
|
|
63
59
|
|
|
64
60
|
### `filter(event)`
|
|
65
61
|
|
|
@@ -79,103 +75,3 @@ Evicts cached identities older than the current sliding window.
|
|
|
79
75
|
### `destroy()`
|
|
80
76
|
|
|
81
77
|
Clears the in-memory cache.
|
|
82
|
-
|
|
83
|
-
## Deployment Test Harness
|
|
84
|
-
|
|
85
|
-
The `test/` directory contains a local runtime harness that simulates:
|
|
86
|
-
|
|
87
|
-
- one collector process running `causal-order`
|
|
88
|
-
- multiple node processes generating events
|
|
89
|
-
- localhost TCP transport between nodes and collector
|
|
90
|
-
- delayed delivery, late arrivals, and duplicate injection
|
|
91
|
-
- per-process logs and JSON run artifacts
|
|
92
|
-
|
|
93
|
-
Useful entry points:
|
|
94
|
-
|
|
95
|
-
```bash
|
|
96
|
-
npm run check
|
|
97
|
-
npm run build
|
|
98
|
-
npm run test:runtime -- --help
|
|
99
|
-
npm run test:runtime -- --duration 1m --steady-for 20s --time-scale 60 --run-name smoke
|
|
100
|
-
npm run test:runtime -- --profile expected-production --duration 1h --steady-for 15m --run-name tuned-test
|
|
101
|
-
npm run summary:latest
|
|
102
|
-
npm run summary:report
|
|
103
|
-
```
|
|
104
|
-
|
|
105
|
-
Artifacts are written under `artifacts/runs/`.
|
|
106
|
-
|
|
107
|
-
Typical files in a run folder:
|
|
108
|
-
|
|
109
|
-
- `summary.json`
|
|
110
|
-
- `run-config.json`
|
|
111
|
-
- `heartbeats.ndjson`
|
|
112
|
-
- `anomalies.ndjson`
|
|
113
|
-
- `lifecycle.ndjson`
|
|
114
|
-
- `orchestrator.log`
|
|
115
|
-
- `collector.stdout.log`
|
|
116
|
-
- `collector.stderr.log`
|
|
117
|
-
- `nodes/*.stdout.log`
|
|
118
|
-
- `nodes/*.stderr.log`
|
|
119
|
-
|
|
120
|
-
## Workload Profiles
|
|
121
|
-
|
|
122
|
-
The harness supports both a built-in default profile and JSON-backed profiles in [profiles/](./profiles).
|
|
123
|
-
|
|
124
|
-
These profiles are workload presets for the local test harness. They shape simulated traffic characteristics such as throughput, node balance, cross-node dependencies, duplicate pressure, ordering preservation, and delivery delay. They are testing scenarios, not authoritative real-world deployment models.
|
|
125
|
-
|
|
126
|
-
Available profiles in this repo:
|
|
127
|
-
|
|
128
|
-
- `expected-production`: production-like baseline you can tune further
|
|
129
|
-
- `expected-production-heavy-cross`: forces much heavier cross-node dependency pressure
|
|
130
|
-
- `expected-production-3way-mesh`: scales the workload toward a 3-node mesh shape
|
|
131
|
-
- `break-the-wire`: harsher partition and latency stress profile
|
|
132
|
-
|
|
133
|
-
Examples:
|
|
134
|
-
|
|
135
|
-
```bash
|
|
136
|
-
npm run test:runtime -- --profile expected-production --duration 10h --steady-for 2h --time-scale 1 --run-name overnight-expected
|
|
137
|
-
npm run test:runtime -- --profile expected-production-heavy-cross --duration 2h --steady-for 30m --time-scale 1 --run-name heavy-cross
|
|
138
|
-
npm run test:runtime -- --profile expected-production-3way-mesh --duration 2h --steady-for 30m --time-scale 1 --run-name mesh
|
|
139
|
-
npm run test:runtime -- --profile break-the-wire --duration 30m --steady-for 5m --time-scale 1 --run-name partition-stress
|
|
140
|
-
```
|
|
141
|
-
|
|
142
|
-
You can also point at a custom JSON file directly:
|
|
143
|
-
|
|
144
|
-
```bash
|
|
145
|
-
npm run test:runtime -- --profile-file profiles/expected-production.json --duration 1h --steady-for 15m --run-name custom-profile
|
|
146
|
-
```
|
|
147
|
-
|
|
148
|
-
The profile files control:
|
|
149
|
-
|
|
150
|
-
- phase rates and chaos multipliers
|
|
151
|
-
- node weighting
|
|
152
|
-
- same-node versus cross-node dependency mix
|
|
153
|
-
- duplicate injection rates
|
|
154
|
-
- ordering preservation probability
|
|
155
|
-
- steady and chaotic delay bands
|
|
156
|
-
|
|
157
|
-
## Notes
|
|
158
|
-
|
|
159
|
-
- The previous `README-pre.md` described an earlier version of this repo with older names and npm script aliases.
|
|
160
|
-
- The repo defines `npm run test:runtime`, `npm run summary:latest`, and `npm run summary:report` for the local harness workflow.
|
|
161
|
-
- The publish workflow uses a generated `publish-dist/` folder with a stripped `package.json`, so local repo scripts are available for development but are not carried in the published npm package.
|
|
162
|
-
|
|
163
|
-
## Publishing
|
|
164
|
-
|
|
165
|
-
Build the publishable package directory:
|
|
166
|
-
|
|
167
|
-
```bash
|
|
168
|
-
npm run prepare:dist
|
|
169
|
-
```
|
|
170
|
-
|
|
171
|
-
Preview the published tarball contents:
|
|
172
|
-
|
|
173
|
-
```bash
|
|
174
|
-
npm run pack:dist
|
|
175
|
-
```
|
|
176
|
-
|
|
177
|
-
Publish the stripped package:
|
|
178
|
-
|
|
179
|
-
```bash
|
|
180
|
-
npm run publish:dist
|
|
181
|
-
```
|
package/package.json
CHANGED
|
@@ -1,8 +1,12 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@causal-order/dedupe",
|
|
3
|
-
"version": "1.0.
|
|
3
|
+
"version": "1.0.2",
|
|
4
4
|
"description": "Deduplication support for causal-order event streams.",
|
|
5
5
|
"license": "MIT",
|
|
6
|
+
"repository": {
|
|
7
|
+
"type": "git",
|
|
8
|
+
"url": "git+https://github.com/GazaliAhmad/causal-order-dedupe.git"
|
|
9
|
+
},
|
|
6
10
|
"type": "module",
|
|
7
11
|
"main": "./src/dedupe.js",
|
|
8
12
|
"types": "./src/dedupe.d.ts",
|