oracular 0.9.0 → 0.11.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +24 -0
- package/README.md +6 -5
- package/dist/cli.js +63 -30
- package/dist/index.cjs +54 -32
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +16 -7
- package/dist/index.d.ts +16 -7
- package/dist/index.js +54 -32
- package/dist/index.js.map +1 -1
- package/dist/presets.cjs +13 -6
- package/dist/presets.cjs.map +1 -1
- package/dist/presets.d.cts +0 -17
- package/dist/presets.d.ts +0 -17
- package/dist/presets.js +13 -6
- package/dist/presets.js.map +1 -1
- package/package.json +1 -1
- package/schema/config.schema.json +4 -3
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,29 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.11.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- [#58](https://github.com/Yuribenjamin/oracular/pull/58) [`d6d1e87`](https://github.com/Yuribenjamin/oracular/commit/d6d1e87e176fc0112384e47fb4782e12d6fa1d1a) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **Learning no longer drifts without outcomes.** The outcome rates a signal is judged by, with it and without it, were Laplace-smoothed, which centres both on 0.5 with one pseudo-hit each: two groups with no outcome at all then differ only by their sizes, so with zero outcomes a common signal drifted negative and a rare one positive, and a weight could change sign on no evidence (a weight of 2 seen in 90 of 100 outcome-free episodes ended at −0.59). Both rates are now shrunk toward the intent's own base rate by two pseudo-episodes, `(hits + 2·q₀) / (n + 2)` with `q₀ = outcomes / episodes`: a signal whose outcome rate equals the base rate learns exactly zero whatever the sample sizes, with no outcome every rate is 0 and no weight moves away from the developer's, and a weight never changes sign without outcomes on at least one side. The association-rule baseline becomes the base rate itself. `tune` fits with the same smoothing, so its weights, intervals and lifts move a little on small samples (a signal seen once with the outcome over a low base rate is worth a little less than before); on large samples the change is in the third decimal. Every learned number in the documentation's generated tables was regenerated. The state format is unchanged, and existing learned counts are kept: only the weight read from them moves.
|
|
8
|
+
|
|
9
|
+
## 0.10.0
|
|
10
|
+
|
|
11
|
+
### Minor Changes
|
|
12
|
+
|
|
13
|
+
- [#56](https://github.com/Yuribenjamin/oracular/pull/56) [`1529d8e`](https://github.com/Yuribenjamin/oracular/commit/1529d8e1568d79959afc7573ba8fa6e7f3e7ff59) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **Configs that cannot work are refused.** `createState`, `validateConfig` and the `config` passed to `deserialize` now throw for a reveal `threshold` of 0 or 1 (`RangeError`, strictly between 0 and 1), a `z` above 10 (`RangeError`; 3.09 is already a one-sided 99.9% bound, and 128 is a misplaced 1.28), and an `outcome` that is also one of the intent's own signals while `resetOnOutcome` is on (`TypeError`: the reset wipes the evidence the signal just added; set `resetOnOutcome: false` to keep such a signal, which then reads as "did it once, will again"). The JSON Schema says the same. Per-call reveal options keep accepting 0 and 1, since a probe such as `evidenceToReveal` at those values is a legitimate question. A stored state whose config predates these checks still loads: `deserialize` reads the stored config without them and replaces it with the one you pass. An intent with no signals stays accepted; it is a prior-only gate. The presets in `oracular/presets` are now frozen through and through, so a change by one caller cannot reach every other; spread one to change a value, as the guide shows.
|
|
14
|
+
|
|
15
|
+
**Size.** The checks, the anchor pin and the duplicate-outcome rule add about 0.2 kB: the core goes from 7.5 kB to 7.72 kB minified and gzipped, under the 7.9 kB ceiling, and the quickstart entry (`createState`, `observe`, `shouldReveal`, `markRevealed`) from 4.43 kB to 4.63 kB, past its 4.5 kB budget, which moves to 4.7 kB.
|
|
16
|
+
|
|
17
|
+
- [#56](https://github.com/Yuribenjamin/oracular/pull/56) [`1529d8e`](https://github.com/Yuribenjamin/oracular/commit/1529d8e1568d79959afc7573ba8fa6e7f3e7ff59) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **A clock that was ahead no longer freezes an intent.** A production audit found that one event stamped in the future (a device clock that was wrong for a moment) held an intent until the clock caught up: decay stopped, because the anchor kept the later time; the open episode never timed out, so learning stopped; and a reveal or outcome stamped in the future kept the gate shut until that date plus the cooldown. Every write now moves its anchor to the event's own time, even when that is earlier: the evidence is held (no decay for a backwards step, as before) and decay resumes from the new time, for `updatedAt`, the learned counts' `learnedAt` and the episode's `lastEventAt`. A reveal or outcome stamped ahead of the clock counts as having happened now: `predict` reports `suppressedUntil` one cooldown after the current time rather than years ahead, and the next event pins the stored time to its own. `markRevealed` follows the same rule and trusts the time it is given, so an earlier `at` moves the cooldown start back. What moves for a clock that only ever runs forward: nothing. What moves for one that jumps backwards: the anchor now follows it, so a log replayed with timestamps out of order decays slightly more than before. `decayFactor` of an infinite gap is now 0, not 1. An episode held under `attribution` with a close stamped in the future is left as it was.
|
|
18
|
+
|
|
19
|
+
- [#56](https://github.com/Yuribenjamin/oracular/pull/56) [`1529d8e`](https://github.com/Yuribenjamin/oracular/commit/1529d8e1568d79959afc7573ba8fa6e7f3e7ff59) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **A repeated outcome counts once.** An outcome rewards the signals seen since the episode started or since the intent's outcome last arrived, and counts the episode toward the base rate. A second outcome event with no signal seen since the first (a duplicate event, or two outcomes back to back) counted another hit episode, so a session with two outcome events inflated the base rate; it now counts nothing, and only `lastOutcomeAt` and the reset of the evidence happen again. Signals seen after an outcome in the same episode are unchanged: they earn a hit if another outcome follows them, and stay neutral at the close, since what they would predict is a second outcome in the same session, which is not what the weight is for. `tune` counts the same way. The state format is unchanged.
|
|
20
|
+
|
|
21
|
+
- [#56](https://github.com/Yuribenjamin/oracular/pull/56) [`1529d8e`](https://github.com/Yuribenjamin/oracular/commit/1529d8e1568d79959afc7573ba8fa6e7f3e7ff59) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **`tune` no longer reports a lift with nothing to compare it to.** A signal seen in every counted episode had its outcome rate compared with a smoothed rate over zero episodes, a made-up 0.5, and the report printed a lift and a 90% interval from it. For such a signal `lift`, `score`, `low` and `high` are now `null` in `--json`, and the text report prints "no contrast" in place of the interval; the weight was already left out as ubiquitous.
|
|
22
|
+
|
|
23
|
+
### Patch Changes
|
|
24
|
+
|
|
25
|
+
- [#56](https://github.com/Yuribenjamin/oracular/pull/56) [`1529d8e`](https://github.com/Yuribenjamin/oracular/commit/1529d8e1568d79959afc7573ba8fa6e7f3e7ff59) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **Two CLI fixes.** `replay` and `tune` no longer refuse a whole config because one intent has a reveal cooldown of 0: that intent is scored over its half-life, the text report says so, and the `--json` reports gain `windowSource` (`option`, `cooldown` or `halfLife`) next to the existing `windowIsCooldown`. `oracular replay --version` and `oracular tune --version` (or `-v`) print the version, as the help already claimed.
|
|
26
|
+
|
|
3
27
|
## 0.9.0
|
|
4
28
|
|
|
5
29
|
### Minor Changes
|
package/README.md
CHANGED
|
@@ -186,15 +186,16 @@ Before any update or reading, the evidence decays by 2^(−Δt / halfLife), wher
|
|
|
186
186
|
|
|
187
187
|
### The user's history corrects your weights
|
|
188
188
|
|
|
189
|
-
Events happen inside **episodes** (sessions). An episode ends when you call `endEpisode`, or on its own after `episodeTimeout` of inactivity. Signals seen in the episode are remembered per intent. When the intent's `outcome` event arrives, each remembered signal gets a **hit**; when the episode ends without it, each gets a **miss**, or, with `attribution` set on the intent, the episode waits and takes the share `2^(−age / halfLife)` of a hit if the outcome arrives inside the window. The engine also counts, per intent, how many episodes ended with and without the outcome: the intent's **base rate**.
|
|
189
|
+
Events happen inside **episodes** (sessions). An episode ends when you call `endEpisode`, or on its own after `episodeTimeout` of inactivity. Signals seen in the episode are remembered per intent. When the intent's `outcome` event arrives, each remembered signal gets a **hit**; when the episode ends without it, each gets a **miss**, or, with `attribution` set on the intent, the episode waits and takes the share `2^(−age / halfLife)` of a hit if the outcome arrives inside the window. The engine also counts, per intent, how many episodes ended with and without the outcome: the intent's **base rate**. Signals seen after an outcome in the same episode count only if another outcome follows them; a repeated outcome with nothing seen since counts nothing.
|
|
190
190
|
|
|
191
191
|
A signal's weight is then a blend of your prior and the learned estimate:
|
|
192
192
|
|
|
193
193
|
```
|
|
194
194
|
nWith = hits + misses episodes with the signal
|
|
195
195
|
nWithout = episodes − nWith episodes without it
|
|
196
|
-
|
|
197
|
-
|
|
196
|
+
q₀ = outcomes / episodes the intent's own base rate
|
|
197
|
+
qWith = (hits + 2·q₀) / (nWith + 2) outcome rate with the signal
|
|
198
|
+
qWithout = (outcomes − hits + 2·q₀) / (nWithout + 2) outcome rate without it
|
|
198
199
|
lift = (qWith − qWithout) / (1 − qWithout) if qWith ≥ qWithout, else (qWith − qWithout) / qWithout → in [−1, 1]
|
|
199
200
|
learned = gain × |w₀| × lift 0 until the signal has history
|
|
200
201
|
nEff = (nWith × nWithout) / (nWith + nWithout) 0 if either side is 0
|
|
@@ -308,7 +309,7 @@ interface RevealOptions {
|
|
|
308
309
|
}
|
|
309
310
|
```
|
|
310
311
|
|
|
311
|
-
Reveal options resolve per call → intent → engine → defaults. Invalid config throws a `TypeError` or `RangeError` with the offending path in the message. `validateConfig(config)` runs those checks without building a state and returns the normalized copy `createState` stores, so a bad weight fails at startup or in a test rather than on first use. For a config kept as JSON, the package ships a [JSON Schema](./schema/config.schema.json) as `oracular/config.schema.json`, also served at <https://yuribenjamin.github.io/oracular/config.schema.json>, for editor completion; `oracular replay` ignores the `$schema` key such a file carries, the engine does not.
|
|
312
|
+
Reveal options resolve per call → intent → engine → defaults. Invalid config throws a `TypeError` or `RangeError` with the offending path in the message, including a config that cannot work: a reveal `threshold` of 0 or 1, a `z` above 10, or an outcome that is also one of the intent's signals while `resetOnOutcome` is on (the reset would wipe what the signal just added; set it to `false` to keep such a signal). A stored state whose config predates these checks still loads. `validateConfig(config)` runs those checks without building a state and returns the normalized copy `createState` stores, so a bad weight fails at startup or in a test rather than on first use. For a config kept as JSON, the package ships a [JSON Schema](./schema/config.schema.json) as `oracular/config.schema.json`, also served at <https://yuribenjamin.github.io/oracular/config.schema.json>, for editor completion; `oracular replay` ignores the `$schema` key such a file carries, the engine does not.
|
|
312
313
|
|
|
313
314
|
### Results
|
|
314
315
|
|
|
@@ -339,7 +340,7 @@ interface SignalExplanation {
|
|
|
339
340
|
|
|
340
341
|
### Time
|
|
341
342
|
|
|
342
|
-
`at` is a timestamp. Every duration (`halfLife`, `cooldown`, `episodeTimeout`, `learning.halfLife`) is in the same unit as `at`. Milliseconds from `Date.now()` are the obvious choice, but the library never calls `Date.now()` itself: you pass time in, which keeps every function deterministic and makes tests trivial. A clock that jumps backwards is tolerated (no decay, no error). When `at` is in milliseconds, `ms.days(7)`, `ms.hours(3)` and `ms.minutes(30)` spell the durations; they are plain multiplications.
|
|
343
|
+
`at` is a timestamp. Every duration (`halfLife`, `cooldown`, `episodeTimeout`, `learning.halfLife`) is in the same unit as `at`. Milliseconds from `Date.now()` are the obvious choice, but the library never calls `Date.now()` itself: you pass time in, which keeps every function deterministic and makes tests trivial. A clock that jumps backwards is tolerated: the evidence is held (no decay, no error) and the anchor moves to the new time, so decay resumes from there. One event stamped ahead of the clock therefore cannot hold an intent until the clock catches up, and a reveal or outcome stamped ahead of it counts as having happened now: its cooldown runs from the current time until the next event pins it there. When `at` is in milliseconds, `ms.days(7)`, `ms.hours(3)` and `ms.minutes(30)` spell the durations; they are plain multiplications.
|
|
343
344
|
|
|
344
345
|
## Persistence
|
|
345
346
|
|
package/dist/cli.js
CHANGED
|
@@ -107,6 +107,14 @@ import {
|
|
|
107
107
|
} from "oracular";
|
|
108
108
|
var DEFAULT_THRESHOLDS = [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9];
|
|
109
109
|
var DEFAULT_THRESHOLD = 0.5;
|
|
110
|
+
function scoringWindow(given, cooldown, halfLife) {
|
|
111
|
+
if (given !== void 0) {
|
|
112
|
+
if (!(given > 0)) throw new RangeError("the scoring window must be greater than 0");
|
|
113
|
+
return { window: given, windowSource: "option" };
|
|
114
|
+
}
|
|
115
|
+
if (cooldown > 0) return { window: cooldown, windowSource: "cooldown" };
|
|
116
|
+
return { window: halfLife, windowSource: "halfLife" };
|
|
117
|
+
}
|
|
110
118
|
function replay(entries, config, options = {}) {
|
|
111
119
|
const base = createState(config);
|
|
112
120
|
const normalized = base.config;
|
|
@@ -135,19 +143,15 @@ function replay(entries, config, options = {}) {
|
|
|
135
143
|
const list = options.thresholds ?? [...DEFAULT_THRESHOLDS, configured];
|
|
136
144
|
const thresholds = [...new Set(list)].sort((a, b) => a - b);
|
|
137
145
|
const cooldown = predict(markRevealed(base, name, 0), name, 0).suppressedUntil ?? 0;
|
|
138
|
-
const window = options.window
|
|
139
|
-
if (!(window > 0)) {
|
|
140
|
-
throw new RangeError(
|
|
141
|
-
options.window === void 0 ? `intent ${JSON.stringify(name)} has a reveal cooldown of 0, so there is no default scoring window; pass --window (for example --window 7d)` : "the scoring window must be greater than 0"
|
|
142
|
-
);
|
|
143
|
-
}
|
|
146
|
+
const { window, windowSource } = scoringWindow(options.window, cooldown, intent.halfLife);
|
|
144
147
|
return {
|
|
145
148
|
name,
|
|
146
149
|
outcome: intent.outcome,
|
|
147
150
|
configured,
|
|
148
151
|
thresholds,
|
|
149
152
|
window,
|
|
150
|
-
|
|
153
|
+
windowSource,
|
|
154
|
+
windowIsCooldown: windowSource === "cooldown"
|
|
151
155
|
};
|
|
152
156
|
});
|
|
153
157
|
const allThresholds = [...new Set(plans.flatMap((p) => p.thresholds))].sort((a, b) => a - b);
|
|
@@ -261,6 +265,7 @@ function replay(entries, config, options = {}) {
|
|
|
261
265
|
intent: plan.name,
|
|
262
266
|
outcome: plan.outcome,
|
|
263
267
|
window: plan.window,
|
|
268
|
+
windowSource: plan.windowSource,
|
|
264
269
|
windowIsCooldown: plan.windowIsCooldown,
|
|
265
270
|
installs,
|
|
266
271
|
thresholds: totals.map((s) => {
|
|
@@ -347,7 +352,7 @@ function table(headers, rows) {
|
|
|
347
352
|
}
|
|
348
353
|
function formatIntent(intent) {
|
|
349
354
|
const window = formatDuration(intent.window);
|
|
350
|
-
const source = intent.
|
|
355
|
+
const source = intent.windowSource === "cooldown" ? "the intent's reveal cooldown" : intent.windowSource === "halfLife" ? "the intent's half-life, since its reveal cooldown is 0" : "as given";
|
|
351
356
|
const out = [];
|
|
352
357
|
out.push(
|
|
353
358
|
`${intent.intent}: outcome ${intent.outcome}, window ${window} (${source}), ${intent.installs} install${intent.installs === 1 ? "" : "s"}`
|
|
@@ -432,6 +437,9 @@ import {
|
|
|
432
437
|
function laplace(successes, total) {
|
|
433
438
|
return (successes + 1) / (total + 2);
|
|
434
439
|
}
|
|
440
|
+
function smoothRate(successes, total, rate) {
|
|
441
|
+
return (successes + 2 * rate) / (total + 2);
|
|
442
|
+
}
|
|
435
443
|
function lift(q, b, type = "normalized-difference") {
|
|
436
444
|
if (type === "absolute-difference") return q - b;
|
|
437
445
|
if (type !== "normalized-difference") {
|
|
@@ -523,14 +531,16 @@ function fitSignal(signal, current, tally, strength, priorEpisodes, liftType) {
|
|
|
523
531
|
const nWith = hits + misses;
|
|
524
532
|
const without = Math.max(0, tally.episodes - nWith);
|
|
525
533
|
const outcomesWithout = Math.min(without, Math.max(0, tally.outcomes - hits));
|
|
526
|
-
const
|
|
527
|
-
const
|
|
528
|
-
const
|
|
529
|
-
const
|
|
530
|
-
const
|
|
534
|
+
const base = tally.episodes === 0 ? 0 : tally.outcomes / tally.episodes;
|
|
535
|
+
const qWith = smoothRate(hits, nWith, base);
|
|
536
|
+
const qWithout = smoothRate(outcomesWithout, without, base);
|
|
537
|
+
const baseline = liftType === "association-rule" ? smoothRate(hits + outcomesWithout, nWith + without, base) : qWithout;
|
|
538
|
+
const noContrast = without === 0;
|
|
539
|
+
const rawLift = noContrast ? null : nWith === 0 ? 0 : lift(qWith, baseline, liftType);
|
|
540
|
+
const score = rawLift === null ? null : nWith === 0 ? 0 : learningScore(rawLift, liftType);
|
|
531
541
|
const contrast = nWith + without === 0 ? 0 : nWith * without / (nWith + without);
|
|
532
542
|
const [qLow, qHigh] = wilson(hits, nWith, Z90);
|
|
533
|
-
const rounded = Math.round(strength * score * 100) / 100;
|
|
543
|
+
const rounded = score === null ? 0 : Math.round(strength * score * 100) / 100;
|
|
534
544
|
const share = tally.episodes === 0 ? 0 : nWith / tally.episodes;
|
|
535
545
|
const ubiquitous = nWith > UBIQUITY_SHARE * tally.episodes;
|
|
536
546
|
return {
|
|
@@ -547,8 +557,8 @@ function fitSignal(signal, current, tally, strength, priorEpisodes, liftType) {
|
|
|
547
557
|
weight: nWith === 0 || ubiquitous || rounded === 0 ? null : rounded,
|
|
548
558
|
share,
|
|
549
559
|
ubiquitous,
|
|
550
|
-
low: strength * learningScore(lift(qLow, baseline, liftType), liftType),
|
|
551
|
-
high: strength * learningScore(lift(qHigh, baseline, liftType), liftType),
|
|
560
|
+
low: noContrast ? null : strength * learningScore(lift(qLow, baseline, liftType), liftType),
|
|
561
|
+
high: noContrast ? null : strength * learningScore(lift(qHigh, baseline, liftType), liftType),
|
|
552
562
|
contrast,
|
|
553
563
|
thin: contrast < priorEpisodes
|
|
554
564
|
};
|
|
@@ -643,13 +653,8 @@ function tune(entries, config, options = {}) {
|
|
|
643
653
|
const grouped = groupByInstall(entries);
|
|
644
654
|
const windows = /* @__PURE__ */ new Map();
|
|
645
655
|
for (const name of fitting) {
|
|
646
|
-
const
|
|
647
|
-
|
|
648
|
-
throw new RangeError(
|
|
649
|
-
options.window === void 0 ? `intent ${JSON.stringify(name)} has a reveal cooldown of 0, so there is no scoring window for the half-life grid; pass --window (for example --window 7d)` : "the scoring window must be greater than 0"
|
|
650
|
-
);
|
|
651
|
-
}
|
|
652
|
-
windows.set(name, window);
|
|
656
|
+
const intent = normalized.intents[name];
|
|
657
|
+
windows.set(name, scoringWindow(options.window, cooldownOf(base, name), intent.halfLife));
|
|
653
658
|
}
|
|
654
659
|
const eventNames = /* @__PURE__ */ new Set();
|
|
655
660
|
for (const entry of entries) if (entry.kind === "event") eventNames.add(entry.name);
|
|
@@ -745,7 +750,7 @@ function tune(entries, config, options = {}) {
|
|
|
745
750
|
signals.filter((f) => f.weight !== null).map((f) => [f.signal, f.weight])
|
|
746
751
|
)
|
|
747
752
|
};
|
|
748
|
-
const window = windows.get(name);
|
|
753
|
+
const { window, windowSource } = windows.get(name);
|
|
749
754
|
const threshold = intent.reveal?.threshold ?? normalized.reveal?.threshold ?? DEFAULT_THRESHOLD;
|
|
750
755
|
const grid = [.../* @__PURE__ */ new Set([...options.halfLives ?? DEFAULT_HALF_LIVES, intent.halfLife])].sort(
|
|
751
756
|
(a, b) => a - b
|
|
@@ -772,6 +777,7 @@ function tune(entries, config, options = {}) {
|
|
|
772
777
|
source: "kept",
|
|
773
778
|
note: "kept: no point in the log could be scored, so the grid says nothing",
|
|
774
779
|
window,
|
|
780
|
+
windowSource,
|
|
775
781
|
curve,
|
|
776
782
|
brierBaseline
|
|
777
783
|
};
|
|
@@ -789,6 +795,7 @@ function tune(entries, config, options = {}) {
|
|
|
789
795
|
source: "kept",
|
|
790
796
|
note: `kept: the grid is flat, the best point (${formatDuration(chosen.halfLife)}, ${best.toFixed(4)}) improves on the current value (${atCurrent.brier.toFixed(4)}) by ${(gain * 100).toFixed(2)}%, less than ${FLAT_GRID_GAIN * 100}%${baseline}`,
|
|
791
797
|
window,
|
|
798
|
+
windowSource,
|
|
792
799
|
curve,
|
|
793
800
|
brierBaseline
|
|
794
801
|
};
|
|
@@ -799,6 +806,7 @@ function tune(entries, config, options = {}) {
|
|
|
799
806
|
source: "grid",
|
|
800
807
|
note: `lowest Brier score over the grid, ${best.toFixed(4)}${baseline}`,
|
|
801
808
|
window,
|
|
809
|
+
windowSource,
|
|
802
810
|
curve,
|
|
803
811
|
brierBaseline
|
|
804
812
|
};
|
|
@@ -874,9 +882,9 @@ function signalTable(fits, priorEpisodes, withCurrent) {
|
|
|
874
882
|
f.signal,
|
|
875
883
|
num(f.weight, 2),
|
|
876
884
|
...withCurrent ? [num(f.current, 2)] : [],
|
|
877
|
-
`${f.low.toFixed(2)} to ${f.high.toFixed(2)}`,
|
|
878
|
-
signed(f.lift, 2),
|
|
879
|
-
...showScore ? [signed(f.score, 2)] : [],
|
|
885
|
+
f.low === null || f.high === null ? "no contrast" : `${f.low.toFixed(2)} to ${f.high.toFixed(2)}`,
|
|
886
|
+
f.lift === null ? "" : signed(f.lift, 2),
|
|
887
|
+
...showScore ? [f.score === null ? "" : signed(f.score, 2)] : [],
|
|
880
888
|
`${f.hits} / ${f.misses}`,
|
|
881
889
|
`${f.outcomesWithout} / ${f.without}`,
|
|
882
890
|
note(f, priorEpisodes)
|
|
@@ -923,7 +931,7 @@ function formatIntent2(fit) {
|
|
|
923
931
|
}
|
|
924
932
|
out.push("");
|
|
925
933
|
out.push(
|
|
926
|
-
`Half-life grid, Brier score against "the outcome follows within ${formatDuration(fit.halfLife.window)}" (lower is better): ${fit.halfLife.curve.map(
|
|
934
|
+
`Half-life grid, Brier score against "the outcome follows within ${formatDuration(fit.halfLife.window)}"${fit.halfLife.windowSource === "halfLife" ? " (the intent's half-life, since its reveal cooldown is 0)" : ""} (lower is better): ${fit.halfLife.curve.map(
|
|
927
935
|
(p) => `${formatDuration(p.halfLife)} ${num(p.brier, 4)}${p.halfLife === fit.halfLife.value && p.brier !== null ? " *" : ""}`
|
|
928
936
|
).join(", ")}`
|
|
929
937
|
);
|
|
@@ -1052,6 +1060,7 @@ Options:
|
|
|
1052
1060
|
the signals
|
|
1053
1061
|
--json print the report as JSON
|
|
1054
1062
|
-h, --help show this help
|
|
1063
|
+
-v, --version print the version
|
|
1055
1064
|
|
|
1056
1065
|
Only shadow periods are counted by default: an episode in which any record was live is left out,
|
|
1057
1066
|
a log with live records gets a warning, and a log with nothing but live records is refused unless
|
|
@@ -1142,16 +1151,29 @@ function commonFlags(out) {
|
|
|
1142
1151
|
},
|
|
1143
1152
|
"-h": () => {
|
|
1144
1153
|
out.help = true;
|
|
1154
|
+
},
|
|
1155
|
+
"--version": (_value, noValue) => {
|
|
1156
|
+
noValue();
|
|
1157
|
+
out.version = true;
|
|
1158
|
+
},
|
|
1159
|
+
"-v": () => {
|
|
1160
|
+
out.version = true;
|
|
1145
1161
|
}
|
|
1146
1162
|
};
|
|
1147
1163
|
}
|
|
1148
1164
|
function requireLogsAndConfig(out, command) {
|
|
1149
|
-
if (out.help) return;
|
|
1165
|
+
if (out.help || out.version) return;
|
|
1150
1166
|
if (out.logs.length === 0) throw new UsageError(`${command} needs at least one log file`);
|
|
1151
1167
|
if (out.config === void 0) throw new UsageError(`${command} needs --config <file>`);
|
|
1152
1168
|
}
|
|
1153
1169
|
function parseReplayArgs(argv) {
|
|
1154
|
-
const out = {
|
|
1170
|
+
const out = {
|
|
1171
|
+
logs: [],
|
|
1172
|
+
includeLive: false,
|
|
1173
|
+
json: false,
|
|
1174
|
+
help: false,
|
|
1175
|
+
version: false
|
|
1176
|
+
};
|
|
1155
1177
|
parseFlags(
|
|
1156
1178
|
argv,
|
|
1157
1179
|
{
|
|
@@ -1171,6 +1193,7 @@ function parseTuneArgs(argv) {
|
|
|
1171
1193
|
includeLive: false,
|
|
1172
1194
|
json: false,
|
|
1173
1195
|
help: false,
|
|
1196
|
+
version: false,
|
|
1174
1197
|
allEvents: false
|
|
1175
1198
|
};
|
|
1176
1199
|
parseFlags(
|
|
@@ -1264,6 +1287,11 @@ async function replayCommand(argv, io) {
|
|
|
1264
1287
|
io.stdout(REPLAY_HELP);
|
|
1265
1288
|
return 0;
|
|
1266
1289
|
}
|
|
1290
|
+
if (args.version) {
|
|
1291
|
+
io.stdout(`${version()}
|
|
1292
|
+
`);
|
|
1293
|
+
return 0;
|
|
1294
|
+
}
|
|
1267
1295
|
const config = await loadCheckedConfig(args, io);
|
|
1268
1296
|
const entries = readLogs(args.logs, io);
|
|
1269
1297
|
checkModes(entries, args.includeLive, io, {
|
|
@@ -1291,6 +1319,11 @@ async function tuneCommand(argv, io) {
|
|
|
1291
1319
|
io.stdout(TUNE_HELP);
|
|
1292
1320
|
return 0;
|
|
1293
1321
|
}
|
|
1322
|
+
if (args.version) {
|
|
1323
|
+
io.stdout(`${version()}
|
|
1324
|
+
`);
|
|
1325
|
+
return 0;
|
|
1326
|
+
}
|
|
1294
1327
|
const config = await loadCheckedConfig(args, io);
|
|
1295
1328
|
const entries = readLogs(args.logs, io);
|
|
1296
1329
|
checkModes(entries, args.includeLive, io, {
|
package/dist/index.cjs
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
// src/math.ts
|
|
4
4
|
function decayFactor(elapsed, halfLife) {
|
|
5
|
-
if (!(elapsed > 0)
|
|
5
|
+
if (!(elapsed > 0)) return 1;
|
|
6
6
|
return 2 ** (-elapsed / halfLife);
|
|
7
7
|
}
|
|
8
8
|
function betaMean(alpha, beta) {
|
|
@@ -79,8 +79,8 @@ function lowerBound(alpha, beta, z, bound) {
|
|
|
79
79
|
function repeatFactor(n) {
|
|
80
80
|
return Math.log1p(1 / n) / Math.LN2;
|
|
81
81
|
}
|
|
82
|
-
function
|
|
83
|
-
return (successes +
|
|
82
|
+
function smoothRate(successes, total, rate) {
|
|
83
|
+
return (successes + 2 * rate) / (total + 2);
|
|
84
84
|
}
|
|
85
85
|
function lift(q, b, type = "normalized-difference") {
|
|
86
86
|
if (type === "absolute-difference") return q - b;
|
|
@@ -138,12 +138,11 @@ function mapRecord(record2, fn) {
|
|
|
138
138
|
var NONE = { hits: 0, misses: 0 };
|
|
139
139
|
function decayLearned(state, learning, at) {
|
|
140
140
|
if (state.learnedAt === null) return { ...state, learnedAt: at };
|
|
141
|
-
const learnedAt = Math.max(state.learnedAt, at);
|
|
142
141
|
const d = learning.halfLife === null ? 1 : decayFactor(at - state.learnedAt, learning.halfLife);
|
|
143
|
-
if (d === 1) return
|
|
142
|
+
if (d === 1) return at === state.learnedAt ? state : { ...state, learnedAt: at };
|
|
144
143
|
return {
|
|
145
144
|
...state,
|
|
146
|
-
learnedAt,
|
|
145
|
+
learnedAt: at,
|
|
147
146
|
learned: mapRecord(state.learned, (r) => ({ hits: r.hits * d, misses: r.misses * d })),
|
|
148
147
|
episodes: { hits: state.episodes.hits * d, misses: state.episodes.misses * d }
|
|
149
148
|
};
|
|
@@ -155,12 +154,13 @@ function signalWeights(state, intent, learning, signal) {
|
|
|
155
154
|
}
|
|
156
155
|
const record2 = getOwn(state.learned, signal) ?? NONE;
|
|
157
156
|
const nWith = record2.hits + record2.misses;
|
|
158
|
-
const qWith = laplace(record2.hits, nWith);
|
|
159
157
|
const totalEpisodes = state.episodes.hits + state.episodes.misses;
|
|
160
158
|
const nWithout = Math.max(0, totalEpisodes - nWith);
|
|
161
159
|
const outcomesWithout = Math.min(nWithout, Math.max(0, state.episodes.hits - record2.hits));
|
|
162
|
-
const
|
|
163
|
-
const
|
|
160
|
+
const base = totalEpisodes === 0 ? 0 : state.episodes.hits / totalEpisodes;
|
|
161
|
+
const qWith = smoothRate(record2.hits, nWith, base);
|
|
162
|
+
const qWithout = smoothRate(outcomesWithout, nWithout, base);
|
|
163
|
+
const baseline = learning.lift === "association-rule" ? smoothRate(record2.hits + outcomesWithout, nWith + nWithout, base) : qWithout;
|
|
164
164
|
const score = learningScore(lift(qWith, baseline, learning.lift), learning.lift);
|
|
165
165
|
const learned = nWith === 0 ? 0 : learning.gain * Math.abs(initial) * score;
|
|
166
166
|
const nEff = nWith + nWithout === 0 ? 0 : nWith * nWithout / (nWith + nWithout);
|
|
@@ -252,13 +252,12 @@ function withContributions(state, intent, contributions) {
|
|
|
252
252
|
function decayIntent(state, intent, at) {
|
|
253
253
|
if (state.updatedAt === null) return { ...state, updatedAt: at };
|
|
254
254
|
const d = decayFactor(at - state.updatedAt, intent.halfLife);
|
|
255
|
-
|
|
256
|
-
if (d === 1) return updatedAt === state.updatedAt ? state : { ...state, updatedAt };
|
|
255
|
+
if (d === 1) return at === state.updatedAt ? state : { ...state, updatedAt: at };
|
|
257
256
|
const contributions = mapRecord(state.contributions, (c) => ({
|
|
258
257
|
alpha: c.alpha * d,
|
|
259
258
|
beta: c.beta * d
|
|
260
259
|
}));
|
|
261
|
-
return withContributions({ ...state, updatedAt }, intent, contributions);
|
|
260
|
+
return withContributions({ ...state, updatedAt: at }, intent, contributions);
|
|
262
261
|
}
|
|
263
262
|
function applySignal(state, intent, signal, weight) {
|
|
264
263
|
const prev = getOwn(state.contributions, signal) ?? { alpha: 0, beta: 0 };
|
|
@@ -275,6 +274,7 @@ var DEFAULT_Z = 1.28;
|
|
|
275
274
|
var DEFAULT_GAIN = 2;
|
|
276
275
|
var DEFAULT_PRIOR_EPISODES = 10;
|
|
277
276
|
var MAX_MAGNITUDE = 1e6;
|
|
277
|
+
var MAX_Z = 10;
|
|
278
278
|
var CONFIG_KEYS = ["intents", "episodeTimeout", "learning", "reveal"];
|
|
279
279
|
var INTENT_KEYS = [
|
|
280
280
|
"prior",
|
|
@@ -359,14 +359,17 @@ function expectKeyName(name, path) {
|
|
|
359
359
|
if (name === "__proto__") typeError(path, 'must not use the reserved name "__proto__"');
|
|
360
360
|
return name;
|
|
361
361
|
}
|
|
362
|
-
function normalizeRevealOptions(value, path) {
|
|
362
|
+
function normalizeRevealOptions(value, path, strict = true) {
|
|
363
363
|
const record2 = expectRecord(value, path);
|
|
364
364
|
expectKeys(record2, REVEAL_KEYS, path);
|
|
365
365
|
const out = {};
|
|
366
366
|
if (record2.threshold !== void 0) {
|
|
367
|
-
out.threshold = expectClosedUnit(record2.threshold, `${path}.threshold`);
|
|
367
|
+
out.threshold = strict ? expectOpenUnit(record2.threshold, `${path}.threshold`) : expectClosedUnit(record2.threshold, `${path}.threshold`);
|
|
368
|
+
}
|
|
369
|
+
if (record2.z !== void 0) {
|
|
370
|
+
out.z = expectNonNegative(record2.z, `${path}.z`);
|
|
371
|
+
if (strict && out.z > MAX_Z) rangeError(`${path}.z`, `must be at most ${MAX_Z}`);
|
|
368
372
|
}
|
|
369
|
-
if (record2.z !== void 0) out.z = expectNonNegative(record2.z, `${path}.z`);
|
|
370
373
|
if (record2.cooldown !== void 0) {
|
|
371
374
|
out.cooldown = expectNonNegative(record2.cooldown, `${path}.cooldown`);
|
|
372
375
|
}
|
|
@@ -396,7 +399,7 @@ function normalizeLearning(value, path) {
|
|
|
396
399
|
}
|
|
397
400
|
return out;
|
|
398
401
|
}
|
|
399
|
-
function normalizeIntentConfig(value, path) {
|
|
402
|
+
function normalizeIntentConfig(value, path, strict = true) {
|
|
400
403
|
const record2 = expectRecord(value, path);
|
|
401
404
|
expectKeys(record2, INTENT_KEYS, path);
|
|
402
405
|
const signalsRecord = expectRecord(record2.signals, `${path}.signals`);
|
|
@@ -438,11 +441,18 @@ function normalizeIntentConfig(value, path) {
|
|
|
438
441
|
if (record2.attribution !== void 0) {
|
|
439
442
|
out.attribution = expectPositive(record2.attribution, `${path}.attribution`);
|
|
440
443
|
}
|
|
441
|
-
if (record2.reveal !== void 0)
|
|
442
|
-
out.reveal = normalizeRevealOptions(record2.reveal, `${path}.reveal
|
|
444
|
+
if (record2.reveal !== void 0) {
|
|
445
|
+
out.reveal = normalizeRevealOptions(record2.reveal, `${path}.reveal`, strict);
|
|
446
|
+
}
|
|
447
|
+
if (strict && out.outcome !== void 0 && hasOwn(signals, out.outcome) && out.resetOnOutcome !== false) {
|
|
448
|
+
typeError(
|
|
449
|
+
`${path}.outcome`,
|
|
450
|
+
"is also one of its signals: the reset on outcome would wipe what it adds; set resetOnOutcome: false to keep it"
|
|
451
|
+
);
|
|
452
|
+
}
|
|
443
453
|
return out;
|
|
444
454
|
}
|
|
445
|
-
function normalizeConfig(value) {
|
|
455
|
+
function normalizeConfig(value, strict = true) {
|
|
446
456
|
const record2 = expectRecord(value, "config");
|
|
447
457
|
expectKeys(record2, CONFIG_KEYS, "config");
|
|
448
458
|
const intentsRecord = expectRecord(record2.intents, "config.intents");
|
|
@@ -453,7 +463,8 @@ function normalizeConfig(value) {
|
|
|
453
463
|
expectKeyName(name, "config.intents");
|
|
454
464
|
intents[name] = normalizeIntentConfig(
|
|
455
465
|
intentsRecord[name],
|
|
456
|
-
`config.intents[${JSON.stringify(name)}]
|
|
466
|
+
`config.intents[${JSON.stringify(name)}]`,
|
|
467
|
+
strict
|
|
457
468
|
);
|
|
458
469
|
}
|
|
459
470
|
const out = { intents };
|
|
@@ -462,8 +473,9 @@ function normalizeConfig(value) {
|
|
|
462
473
|
}
|
|
463
474
|
if (record2.learning !== void 0)
|
|
464
475
|
out.learning = normalizeLearning(record2.learning, "config.learning");
|
|
465
|
-
if (record2.reveal !== void 0)
|
|
466
|
-
out.reveal = normalizeRevealOptions(record2.reveal, "config.reveal");
|
|
476
|
+
if (record2.reveal !== void 0) {
|
|
477
|
+
out.reveal = normalizeRevealOptions(record2.reveal, "config.reveal", strict);
|
|
478
|
+
}
|
|
467
479
|
return out;
|
|
468
480
|
}
|
|
469
481
|
function normalizeExplainOptions(value, path) {
|
|
@@ -493,12 +505,12 @@ function splitEvaluateOptions(value, path) {
|
|
|
493
505
|
else explain2[key] = v;
|
|
494
506
|
}
|
|
495
507
|
return {
|
|
496
|
-
reveal: normalizeRevealOptions(reveal, path),
|
|
508
|
+
reveal: normalizeRevealOptions(reveal, path, false),
|
|
497
509
|
explain: normalizeExplainOptions(explain2, path)
|
|
498
510
|
};
|
|
499
511
|
}
|
|
500
512
|
function resolveReveal(config, intent, opts, normalized = false) {
|
|
501
|
-
const o = opts === void 0 ? {} : normalized ? opts : normalizeRevealOptions(opts, "options");
|
|
513
|
+
const o = opts === void 0 ? {} : normalized ? opts : normalizeRevealOptions(opts, "options", false);
|
|
502
514
|
return {
|
|
503
515
|
threshold: o.threshold ?? intent.reveal?.threshold ?? config?.reveal?.threshold ?? DEFAULT_THRESHOLD,
|
|
504
516
|
z: o.z ?? intent.reveal?.z ?? config?.reveal?.z ?? DEFAULT_Z,
|
|
@@ -536,11 +548,7 @@ function timedOutAt(episode, timeout, at) {
|
|
|
536
548
|
return at - episode.lastEventAt > timeout ? episode.lastEventAt + timeout : null;
|
|
537
549
|
}
|
|
538
550
|
function extendEpisode(episode, at) {
|
|
539
|
-
return {
|
|
540
|
-
startedAt: episode.startedAt ?? at,
|
|
541
|
-
lastEventAt: episode.lastEventAt === null ? at : Math.max(episode.lastEventAt, at),
|
|
542
|
-
intents: episode.intents
|
|
543
|
-
};
|
|
551
|
+
return { startedAt: episode.startedAt ?? at, lastEventAt: at, intents: episode.intents };
|
|
544
552
|
}
|
|
545
553
|
function episodeRecord(episode, intent) {
|
|
546
554
|
return getOwn(episode.intents, intent) ?? { seen: [], counts: {}, resolved: false };
|
|
@@ -598,7 +606,7 @@ function assessIntent(state, name, intent, learning, at) {
|
|
|
598
606
|
}
|
|
599
607
|
function cooldownEnd(since, cooldown, at) {
|
|
600
608
|
if (since === null) return null;
|
|
601
|
-
const end = since + cooldown;
|
|
609
|
+
const end = Math.min(since, at) + cooldown;
|
|
602
610
|
return at < end ? end : null;
|
|
603
611
|
}
|
|
604
612
|
function buildPrediction(is, name, reveal, at) {
|
|
@@ -726,6 +734,7 @@ function observe(state, event) {
|
|
|
726
734
|
let is = getIntentState(s, intentName, intent);
|
|
727
735
|
is = decayIntent(is, intent, at);
|
|
728
736
|
is = settlePending(is, intent, learning, at);
|
|
737
|
+
is = pinAnchors(is, at);
|
|
729
738
|
if (isSignal) {
|
|
730
739
|
const { effective } = signalWeights(is, intent, learning, name);
|
|
731
740
|
const n = occurrences(episode, intentName, name) + 1;
|
|
@@ -735,7 +744,8 @@ function observe(state, event) {
|
|
|
735
744
|
if (!isOutcome) episode = markSeen(episode, intentName, name);
|
|
736
745
|
}
|
|
737
746
|
if (isOutcome) {
|
|
738
|
-
|
|
747
|
+
const record2 = episodeRecord(episode, intentName);
|
|
748
|
+
if (!record2.resolved || record2.seen.length > 0) is = recordEpisode(is, record2.seen, 1);
|
|
739
749
|
is = creditPending(is, intent, at);
|
|
740
750
|
episode = markResolved(episode, intentName);
|
|
741
751
|
is = { ...is, lastOutcomeAt: at };
|
|
@@ -745,6 +755,16 @@ function observe(state, event) {
|
|
|
745
755
|
}
|
|
746
756
|
return { ...s, intents, episode };
|
|
747
757
|
}
|
|
758
|
+
function pinAnchors(is, at) {
|
|
759
|
+
const reveal = is.lastRevealedAt !== null && is.lastRevealedAt > at;
|
|
760
|
+
const outcome = is.lastOutcomeAt !== null && is.lastOutcomeAt > at;
|
|
761
|
+
if (!reveal && !outcome) return is;
|
|
762
|
+
return {
|
|
763
|
+
...is,
|
|
764
|
+
lastRevealedAt: reveal ? at : is.lastRevealedAt,
|
|
765
|
+
lastOutcomeAt: outcome ? at : is.lastOutcomeAt
|
|
766
|
+
};
|
|
767
|
+
}
|
|
748
768
|
function endEpisode(state, at) {
|
|
749
769
|
const s = requireState(state);
|
|
750
770
|
const t = expectTimestamp(at, "at");
|
|
@@ -1062,7 +1082,9 @@ function parseState(value) {
|
|
|
1062
1082
|
}
|
|
1063
1083
|
return {
|
|
1064
1084
|
version: STATE_VERSION,
|
|
1065
|
-
|
|
1085
|
+
// Without the strict checks: a config this release would refuse at createState still loads
|
|
1086
|
+
// when it is only stored, and the config passed in replaces it anyway.
|
|
1087
|
+
config: normalizeConfig(r.config, false),
|
|
1066
1088
|
intents: mapOwn(r.intents, "state.intents", intentState),
|
|
1067
1089
|
episode: episodeState(r.episode, "state.episode")
|
|
1068
1090
|
};
|