oracular 0.5.2 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +22 -0
- package/README.md +27 -10
- package/dist/cli.js +11 -1
- package/dist/index.cjs +142 -51
- package/dist/index.cjs.map +1 -1
- package/dist/index.d.cts +114 -17
- package/dist/index.d.ts +114 -17
- package/dist/index.js +142 -52
- package/dist/index.js.map +1 -1
- package/dist/inspect.cjs +67 -0
- package/dist/inspect.cjs.map +1 -0
- package/dist/inspect.d.cts +143 -0
- package/dist/inspect.d.ts +143 -0
- package/dist/inspect.js +64 -0
- package/dist/inspect.js.map +1 -0
- package/dist/presets.cjs +86 -0
- package/dist/presets.cjs.map +1 -0
- package/dist/presets.d.cts +129 -0
- package/dist/presets.d.ts +129 -0
- package/dist/presets.js +83 -0
- package/dist/presets.js.map +1 -0
- package/package.json +33 -1
- package/schema/config.schema.json +4 -0
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,27 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 0.7.0
|
|
4
|
+
|
|
5
|
+
### Minor Changes
|
|
6
|
+
|
|
7
|
+
- [#36](https://github.com/Yuribenjamin/oracular/pull/36) [`a585808`](https://github.com/Yuribenjamin/oracular/commit/a58580846d39fa14269096fa3e4f0891749eee88) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **Inspect.** `inspect(state, at, opts?)` in the main entry returns every intent's prediction, gate and explanation as one object, in config order. `oracular/inspect` renders that snapshot: `renderInspector` as an HTML fragment with inline styles and every string escaped, `mountInspector` into anything with an `innerHTML`, with `update` and `unmount`; neither touches a DOM API or sets a timer, and neither carries engine code. The subpath is experimental, as the stability page says: its markup and options may change in a minor. The types `Snapshot` and `IntentSnapshot` are exported from the main entry.
|
|
8
|
+
|
|
9
|
+
- [#36](https://github.com/Yuribenjamin/oracular/pull/36) [`a585808`](https://github.com/Yuribenjamin/oracular/commit/a58580846d39fa14269096fa3e4f0891749eee88) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **Presets.** `oracular/presets` exports `presets`, four plain `IntentConfig` objects to spread into your config and override: `willBuy`, tuned by `oracular tune` on a public e-commerce clickstream of 2.7 million events (one shop, May to September 2015, 4.8% of sessions ending with an order; prior 0.048, strength 4, `cart.added` 2.3, half-life one hour, 181,080 sessions behind it), and `upgrade`, `stuck` and `churn`, the documentation's running examples, hand-written and labelled so. `presetSources` carries each preset's provenance as data, and the types `PresetName` and `PresetSource` are exported from the main entry.
|
|
10
|
+
|
|
11
|
+
### Patch Changes
|
|
12
|
+
|
|
13
|
+
- [#36](https://github.com/Yuribenjamin/oracular/pull/36) [`a585808`](https://github.com/Yuribenjamin/oracular/commit/a58580846d39fa14269096fa3e4f0891749eee88) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **Rollback.** `deserialize` reads a state one version newer than `STATE_VERSION` as the current one, ignoring the fields it does not know, so a rollback by one release keeps every user's history. The rule that allows it, a state version bump only adds fields, is now a commitment on the new stability page, together with what counts as public API, how versions are read before 1.0, the deprecation rule, the state-format guarantee, the 1.0 criteria and their status, and a 7 kB ceiling for the core. Two versions ahead still throws the `RangeError`, and `migrate` is still consulted for it. From now on the changeset of every state version bump says what an older reader silently loses; this release bumps nothing, so a 0.6 reader loses nothing but was never able to read ahead.
|
|
14
|
+
|
|
15
|
+
## 0.6.0
|
|
16
|
+
|
|
17
|
+
### Minor Changes
|
|
18
|
+
|
|
19
|
+
- [#32](https://github.com/Yuribenjamin/oracular/pull/32) [`6951d70`](https://github.com/Yuribenjamin/oracular/commit/6951d70b4ff1e654ab17a61357d882fd82240968) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **Cross-episode attribution.** `IntentConfig.attribution` is how long a closed episode's signals wait for the outcome before they take a miss. Until now a signal earned a hit only when the outcome arrived in the same episode, so for an intent that builds over days the sessions that built it took misses and the session where the user acted got all the credit. With a window set, an episode that ends without the outcome is held instead of counted; when the outcome arrives inside the window, each held episode gets the share `2^(−age / halfLife)` of a hit that its evidence still holds, and the rest of a miss, so every episode still counts exactly once and the contrast that learning rests on is unchanged; when the window passes, the episode is a miss recorded at the moment the window ended. Reads settle what has expired in what they show and store nothing. Off unless set: every state and estimate without `attribution` is exactly what it was.
|
|
20
|
+
|
|
21
|
+
**State version 3.** `IntentState.pending` lists the episodes waiting, at most 32 per intent (past that the oldest is a miss at once). `deserialize` upgrades version 1 and 2 states on its own, adding an empty list and changing no number; oracular 0.5.x cannot read a version 3 state, so the rollback advice stays: catch and start from `createState(config)`. `PendingEpisode` is exported as a type; the config JSON Schema gains `attribution`.
|
|
22
|
+
|
|
23
|
+
`oracular tune` counts each episode's own verdict, with the window off while counting, keeps it on for the half-life grid, and keeps whatever `attribution` you set in the printed config.
|
|
24
|
+
|
|
3
25
|
## 0.5.2
|
|
4
26
|
|
|
5
27
|
### Patch Changes
|
package/README.md
CHANGED
|
@@ -64,9 +64,21 @@ With these numbers the offer fires after roughly six strong signals across separ
|
|
|
64
64
|
|
|
65
65
|
That block is copied verbatim into the test suite, so it always runs.
|
|
66
66
|
|
|
67
|
+
## The workflow: shadow, tune, ship, learn
|
|
68
|
+
|
|
69
|
+
The weights above are a guess, and the point of oracular is that you do not have to live with a guess. The intended path:
|
|
70
|
+
|
|
71
|
+
1. **Start from a preset or a guess.** `oracular/presets` has `willBuy`, `upgrade`, `stuck` and `churn` as plain configs to spread and override; each says whether its numbers were fitted or written by hand.
|
|
72
|
+
2. **Run in shadow mode.** The engine computes every decision and your app records it with `createRecord`, one JSON line per event, and shows nothing yet.
|
|
73
|
+
3. **Tune.** `npx oracular tune shadow.ndjson --config oracular.config.json` reads what the engine itself counted across your users and prints the config to ship: the base rate as the prior, each signal's measured lift as its weight, the half-life that predicts the log best, with sample sizes and intervals. `npx oracular replay` scores any config on the same log before you commit to it.
|
|
74
|
+
4. **Ship, and let it learn.** The fitted config is the population's answer; on each device the engine corrects it from that user's own outcomes.
|
|
75
|
+
|
|
76
|
+
On a public clickstream of 2.7 million events, a hand-written config scored worse than always guessing the base rate, and the tuned one 10% better, with a reveal followed by an order six times as often as chance. Against the rule a skeptic would write, reveal on the third cart add, it ties for first-time visitors, as it must, and for returning ones catches 43% more orders at three points higher precision, because it has learned who fills carts without buying. The [case study](https://yuribenjamin.github.io/oracular/guide/case-study-clickstream) has every number; the [Shadow mode guide](https://yuribenjamin.github.io/oracular/guide/shadow-mode) has the workflow.
|
|
77
|
+
|
|
67
78
|
## Contents
|
|
68
79
|
|
|
69
80
|
- [Why oracular](#why-oracular)
|
|
81
|
+
- [The workflow: shadow, tune, ship, learn](#the-workflow-shadow-tune-ship-learn)
|
|
70
82
|
- [How it works](#how-it-works)
|
|
71
83
|
- [Examples](#examples)
|
|
72
84
|
- [Using with React](#using-with-react)
|
|
@@ -107,7 +119,7 @@ Before any update or reading, the evidence decays by 2^(−Δt / halfLife), wher
|
|
|
107
119
|
|
|
108
120
|
### The user's history corrects your weights
|
|
109
121
|
|
|
110
|
-
Events happen inside **episodes** (sessions). An episode ends when you call `endEpisode`, or on its own after `episodeTimeout` of inactivity. Signals seen in the episode are remembered per intent. When the intent's `outcome` event arrives, each remembered signal gets a **hit**; when the episode ends without it, each gets a **miss
|
|
122
|
+
Events happen inside **episodes** (sessions). An episode ends when you call `endEpisode`, or on its own after `episodeTimeout` of inactivity. Signals seen in the episode are remembered per intent. When the intent's `outcome` event arrives, each remembered signal gets a **hit**; when the episode ends without it, each gets a **miss**, or, with `attribution` set on the intent, the episode waits and takes the share `2^(−age / halfLife)` of a hit if the outcome arrives inside the window. The engine also counts, per intent, how many episodes ended with and without the outcome: the intent's **base rate**.
|
|
111
123
|
|
|
112
124
|
A signal's weight is then a blend of your prior and the learned estimate:
|
|
113
125
|
|
|
@@ -188,12 +200,12 @@ const churn = {
|
|
|
188
200
|
### Ad placement
|
|
189
201
|
|
|
190
202
|
```ts
|
|
203
|
+
import { presets } from 'oracular/presets';
|
|
204
|
+
|
|
205
|
+
// The tuned preset (prior 0.048, strength 4, cart.added 2.3, from a real clickstream), plus your own signals.
|
|
191
206
|
const willBuy = {
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
halfLife: 3 * 24 * 60 * 60 * 1000,
|
|
195
|
-
signals: { 'product.viewed': 1, 'cart.added': 3, 'wishlist.added': 2, 'cart.emptied': -3 },
|
|
196
|
-
outcome: 'checkout.completed',
|
|
207
|
+
...presets.willBuy,
|
|
208
|
+
signals: { ...presets.willBuy.signals, 'wishlist.added': 2, 'cart.emptied': -3 },
|
|
197
209
|
};
|
|
198
210
|
// shouldReveal(state, 'willBuy', now) → show the sponsored placement, then markRevealed.
|
|
199
211
|
```
|
|
@@ -244,6 +256,7 @@ shouldReveal(state, intent: string, at: number, opts?: RevealOptions): boolean
|
|
|
244
256
|
markRevealed(state, intent: string, at: number): EngineState
|
|
245
257
|
explain(state, intent: string, at: number, opts?: { nonZero?: boolean; limit?: number }): SignalExplanation[]
|
|
246
258
|
evaluate(state, intent: string, at: number, opts?): { prediction, reveal, explanation } // all three, one decay pass
|
|
259
|
+
inspect(state, at: number, opts?): { at, intents: [{ intent, prediction, reveal, explanation }] } // evaluate for every intent
|
|
247
260
|
evidenceToReveal(intentConfig, opts?): number // pseudo-counts at which shouldReveal first becomes true
|
|
248
261
|
episodeClosed(prev, next): boolean // did an episode close between two states?
|
|
249
262
|
createRecord(state, event, { installId, mode }): EventRecord // one recorder line: the event and what was decided
|
|
@@ -252,7 +265,7 @@ serialize(state): string
|
|
|
252
265
|
deserialize(json: string, opts?: { config?: EngineConfig; migrate?: Migrate }): EngineState
|
|
253
266
|
validateConfig(config: unknown): EngineConfig // the checks createState runs, without a state; returns the normalized copy
|
|
254
267
|
ms.seconds(n), ms.minutes(n), ms.hours(n), ms.days(n), ms.weeks(n) // durations in milliseconds
|
|
255
|
-
STATE_VERSION //
|
|
268
|
+
STATE_VERSION // 3
|
|
256
269
|
RECORD_VERSION // 1
|
|
257
270
|
```
|
|
258
271
|
|
|
@@ -278,6 +291,7 @@ interface IntentConfig {
|
|
|
278
291
|
outcome?: string; // event that confirms the intent
|
|
279
292
|
resetOnOutcome?: boolean; // default true
|
|
280
293
|
saturate?: boolean; // repeats within an episode add less; default: episodeTimeout is set
|
|
294
|
+
attribution?: number; // how long a closed episode waits for the outcome before it is a miss; off if omitted
|
|
281
295
|
reveal?: RevealOptions;
|
|
282
296
|
}
|
|
283
297
|
|
|
@@ -337,7 +351,7 @@ const state = deserialize(saved, { config });
|
|
|
337
351
|
|
|
338
352
|
The new config is validated and swapped in. Intents and signals you no longer declare are dropped (α and β are recomputed from what remains), newly declared intents start at their prior, and everything learned about signals that still exist is kept.
|
|
339
353
|
|
|
340
|
-
States saved by an older version of this library are upgraded on load: `deserialize` migrates the state
|
|
354
|
+
States saved by an older version of this library are upgraded on load: `deserialize` migrates the state shapes written by oracular 0.1.0 to 0.5.x (`STATE_VERSION` 1 and 2) to the current one on its own. A `migrate` function is consulted only for a version the library does not know, such as a state written by a newer oracular before a rollback; without one, that throws a `RangeError`:
|
|
341
355
|
|
|
342
356
|
```ts
|
|
343
357
|
deserialize(saved, {
|
|
@@ -363,6 +377,8 @@ explain(state, 'upgrade', Date.now());
|
|
|
363
377
|
|
|
364
378
|
Rows are sorted by the size of their impact. `limit.hit` has no history yet, so its effective weight is exactly the initial one. `pricing.viewed` was followed by an upgrade in six of its seven sessions while the three sessions without it saw one upgrade, so it learned a weight above the initial 3. `export.clicked` appeared in every session: it has no counterfactual, so whatever its hit rate, its weight stands at exactly the initial 1 until a session happens without it. See [Episodes & learning](https://yuribenjamin.github.io/oracular/guide/concepts/episodes-and-learning) for why learning needs contrast.
|
|
365
379
|
|
|
380
|
+
`inspect(state, now)` returns all of that for every intent as one object, and `oracular/inspect` renders it as a live, framework-free panel: `mountInspector(element, () => inspect(state, Date.now()))`, then `update()` after each `observe`. See [Inspect](https://yuribenjamin.github.io/oracular/guide/inspect).
|
|
381
|
+
|
|
366
382
|
## Shadow mode, replay and tune
|
|
367
383
|
|
|
368
384
|
Run oracular silently first. Record what it decides, export the log, and measure a config offline before any reveal is shown:
|
|
@@ -382,7 +398,7 @@ npx oracular replay shadow.ndjson --config oracular.config.json
|
|
|
382
398
|
npx oracular tune shadow.ndjson --config oracular.config.json
|
|
383
399
|
```
|
|
384
400
|
|
|
385
|
-
`tune` seeds a config from the same log: grouped by install, the base rate becomes the prior, the spread of per-install base rates the prior strength, each signal's lift over the episodes without it its weight, and the half-life that predicts the log best wins a grid search scored by `replay`. The counts come from the engine itself, so tune cannot disagree with what a device would learn. It prints sample sizes and a 90% interval per weight, and the config to paste; the engine corrects it per user from there. Three rules learned from a real clickstream keep it honest: a signal seen in nearly every episode is left out, a flat grid keeps your half-life, and the report says when an intent still does not beat its base rate. The [Shadow mode guide](https://yuribenjamin.github.io/oracular/guide/shadow-mode) has the workflow, the record format and both commands.
|
|
401
|
+
`tune` seeds a config from the same log: grouped by install, the base rate becomes the prior, the spread of per-install base rates the prior strength, each signal's lift over the episodes without it its weight, and the half-life that predicts the log best wins a grid search scored by `replay`. The counts come from the engine itself, so tune cannot disagree with what a device would learn. It prints sample sizes and a 90% interval per weight, and the config to paste; the engine corrects it per user from there. Three rules learned from a real clickstream keep it honest: a signal seen in nearly every episode is left out, a flat grid keeps your half-life, and the report says when an intent still does not beat its base rate. The [Shadow mode guide](https://yuribenjamin.github.io/oracular/guide/shadow-mode) has the workflow, the record format and both commands; the [case study](https://yuribenjamin.github.io/oracular/guide/case-study-clickstream) runs both on a public clickstream of 2.7 million events.
|
|
386
402
|
|
|
387
403
|
## Design rules
|
|
388
404
|
|
|
@@ -390,10 +406,11 @@ npx oracular tune shadow.ndjson --config oracular.config.json
|
|
|
390
406
|
- **Portable.** No DOM, Node or React Native specific APIs; the source compiles against the ES2020 library alone. ESM and CommonJS builds with type declarations.
|
|
391
407
|
- **Private.** Nothing leaves the device because nothing *can*: there is no I/O in the library.
|
|
392
408
|
- **Honest about uncertainty.** `shouldReveal` needs both a high mean and a high lower bound.
|
|
409
|
+
- **Committed.** What counts as public API, how the state format may change before 1.0 and what 1.0 will promise are written down in [Stability and 1.0](https://yuribenjamin.github.io/oracular/guide/stability).
|
|
393
410
|
|
|
394
411
|
## Roadmap
|
|
395
412
|
|
|
396
|
-
|
|
413
|
+
The 0.4 theme, **learning that matches how people behave**, is complete: cross-episode attribution (`attribution` on an intent, so a session's signals earn a share of an outcome days later), `npx oracular tune` to seed a config from a shadow log, presets, and small ergonomics (`validateConfig`, `ms`, a JSON Schema for the config). 0.3 shipped shadow mode, `npx oracular replay` and per-episode saturation. Next is the road to 1.0, whose criteria are on [Stability and 1.0](https://yuribenjamin.github.io/oracular/guide/stability). Framework bindings wait for a third integration. The full list, with the reasoning and what was declined, is on the [roadmap page](https://yuribenjamin.github.io/oracular/guide/roadmap).
|
|
397
414
|
|
|
398
415
|
## Contributing
|
|
399
416
|
|
package/dist/cli.js
CHANGED
|
@@ -575,6 +575,16 @@ function withoutLearnedDecay(config) {
|
|
|
575
575
|
);
|
|
576
576
|
return { ...config, learning };
|
|
577
577
|
}
|
|
578
|
+
function withoutAttribution(config) {
|
|
579
|
+
const intents = Object.fromEntries(
|
|
580
|
+
Object.entries(config.intents).map(([name, intent]) => {
|
|
581
|
+
if (intent.attribution === void 0) return [name, intent];
|
|
582
|
+
const { attribution: _attribution, ...rest } = intent;
|
|
583
|
+
return [name, rest];
|
|
584
|
+
})
|
|
585
|
+
);
|
|
586
|
+
return { ...config, intents };
|
|
587
|
+
}
|
|
578
588
|
function cooldownOf(base, name) {
|
|
579
589
|
return predict2(markRevealed2(base, name, 0), name, 0).suppressedUntil ?? 0;
|
|
580
590
|
}
|
|
@@ -613,7 +623,7 @@ function tune(entries, config, options = {}) {
|
|
|
613
623
|
const candidatesOf = (intent) => options.allEvents === true ? [...eventNames].filter(
|
|
614
624
|
(name) => name !== "__proto__" && name !== intent.outcome && !Object.hasOwn(intent.signals, name)
|
|
615
625
|
) : [];
|
|
616
|
-
let countConfig = withoutLearnedDecay(normalized);
|
|
626
|
+
let countConfig = withoutAttribution(withoutLearnedDecay(normalized));
|
|
617
627
|
const plans = fitting.map((name) => {
|
|
618
628
|
const intent = normalized.intents[name];
|
|
619
629
|
const candidates = candidatesOf(intent);
|
package/dist/index.cjs
CHANGED
|
@@ -49,6 +49,86 @@ function mapRecord(record2, fn) {
|
|
|
49
49
|
return out;
|
|
50
50
|
}
|
|
51
51
|
|
|
52
|
+
// src/learning.ts
|
|
53
|
+
var NONE = { hits: 0, misses: 0 };
|
|
54
|
+
function decayLearned(state, learning, at) {
|
|
55
|
+
if (state.learnedAt === null) return { ...state, learnedAt: at };
|
|
56
|
+
const learnedAt = Math.max(state.learnedAt, at);
|
|
57
|
+
const d = learning.halfLife === null ? 1 : decayFactor(at - state.learnedAt, learning.halfLife);
|
|
58
|
+
if (d === 1) return learnedAt === state.learnedAt ? state : { ...state, learnedAt };
|
|
59
|
+
return {
|
|
60
|
+
...state,
|
|
61
|
+
learnedAt,
|
|
62
|
+
learned: mapRecord(state.learned, (r) => ({ hits: r.hits * d, misses: r.misses * d })),
|
|
63
|
+
episodes: { hits: state.episodes.hits * d, misses: state.episodes.misses * d }
|
|
64
|
+
};
|
|
65
|
+
}
|
|
66
|
+
function signalWeights(state, intent, learning, signal) {
|
|
67
|
+
const initial = getOwn(intent.signals, signal);
|
|
68
|
+
if (initial === void 0) {
|
|
69
|
+
throw new RangeError(`oracular: ${JSON.stringify(signal)} is not a signal of this intent`);
|
|
70
|
+
}
|
|
71
|
+
const record2 = getOwn(state.learned, signal) ?? NONE;
|
|
72
|
+
const nWith = record2.hits + record2.misses;
|
|
73
|
+
const qWith = laplace(record2.hits, nWith);
|
|
74
|
+
const totalEpisodes = state.episodes.hits + state.episodes.misses;
|
|
75
|
+
const nWithout = Math.max(0, totalEpisodes - nWith);
|
|
76
|
+
const outcomesWithout = Math.min(nWithout, Math.max(0, state.episodes.hits - record2.hits));
|
|
77
|
+
const qWithout = laplace(outcomesWithout, nWithout);
|
|
78
|
+
const learned = nWith === 0 ? 0 : learning.gain * Math.abs(initial) * lift(qWith, qWithout);
|
|
79
|
+
const nEff = nWith + nWithout === 0 ? 0 : nWith * nWithout / (nWith + nWithout);
|
|
80
|
+
const k = learning.priorEpisodes;
|
|
81
|
+
const effective = (k * initial + nEff * learned) / (k + nEff);
|
|
82
|
+
return { initial, learned, effective, hits: record2.hits, misses: record2.misses };
|
|
83
|
+
}
|
|
84
|
+
function recordEpisode(state, signals, credit) {
|
|
85
|
+
const miss = 1 - credit;
|
|
86
|
+
const learned = { ...state.learned };
|
|
87
|
+
for (const signal of signals) {
|
|
88
|
+
const r = getOwn(learned, signal) ?? NONE;
|
|
89
|
+
learned[signal] = { hits: r.hits + credit, misses: r.misses + miss };
|
|
90
|
+
}
|
|
91
|
+
const episodes = { hits: state.episodes.hits + credit, misses: state.episodes.misses + miss };
|
|
92
|
+
return { ...state, learned, episodes };
|
|
93
|
+
}
|
|
94
|
+
|
|
95
|
+
// src/attribution.ts
|
|
96
|
+
var MAX_PENDING = 32;
|
|
97
|
+
function expiry(episode, intent) {
|
|
98
|
+
return episode.endedAt + (intent.attribution ?? 0);
|
|
99
|
+
}
|
|
100
|
+
function settlePending(state, intent, learning, at) {
|
|
101
|
+
let is = state;
|
|
102
|
+
let settled = 0;
|
|
103
|
+
for (const episode of state.pending) {
|
|
104
|
+
const end = expiry(episode, intent);
|
|
105
|
+
if (!(end < at)) break;
|
|
106
|
+
is = recordEpisode(decayLearned(is, learning, end), episode.seen, 0);
|
|
107
|
+
settled++;
|
|
108
|
+
}
|
|
109
|
+
if (settled > 0) is = { ...is, pending: state.pending.slice(settled) };
|
|
110
|
+
return decayLearned(is, learning, at);
|
|
111
|
+
}
|
|
112
|
+
function deferEpisode(state, seen, endedAt) {
|
|
113
|
+
const pending = insert(state.pending, { endedAt, seen: [...seen] });
|
|
114
|
+
if (pending.length <= MAX_PENDING) return { ...state, pending };
|
|
115
|
+
const [oldest, ...rest] = pending;
|
|
116
|
+
return { ...recordEpisode(state, oldest.seen, 0), pending: rest };
|
|
117
|
+
}
|
|
118
|
+
function insert(pending, episode) {
|
|
119
|
+
let i = pending.length;
|
|
120
|
+
while (i > 0 && pending[i - 1].endedAt > episode.endedAt) i--;
|
|
121
|
+
return [...pending.slice(0, i), episode, ...pending.slice(i)];
|
|
122
|
+
}
|
|
123
|
+
function creditPending(state, intent, at) {
|
|
124
|
+
if (state.pending.length === 0) return state;
|
|
125
|
+
let is = state;
|
|
126
|
+
for (const episode of state.pending) {
|
|
127
|
+
is = recordEpisode(is, episode.seen, decayFactor(at - episode.endedAt, intent.halfLife));
|
|
128
|
+
}
|
|
129
|
+
return { ...is, pending: [] };
|
|
130
|
+
}
|
|
131
|
+
|
|
52
132
|
// src/intent.ts
|
|
53
133
|
function priorCounts(intent) {
|
|
54
134
|
return {
|
|
@@ -65,6 +145,7 @@ function initIntent(intent) {
|
|
|
65
145
|
contributions: {},
|
|
66
146
|
learned: {},
|
|
67
147
|
episodes: { hits: 0, misses: 0 },
|
|
148
|
+
pending: [],
|
|
68
149
|
learnedAt: null,
|
|
69
150
|
lastRevealedAt: null,
|
|
70
151
|
lastOutcomeAt: null,
|
|
@@ -101,48 +182,6 @@ function resetToPrior(state, intent) {
|
|
|
101
182
|
return withContributions(state, intent, {});
|
|
102
183
|
}
|
|
103
184
|
|
|
104
|
-
// src/learning.ts
|
|
105
|
-
var NONE = { hits: 0, misses: 0 };
|
|
106
|
-
function decayLearned(state, learning, at) {
|
|
107
|
-
if (state.learnedAt === null) return { ...state, learnedAt: at };
|
|
108
|
-
const learnedAt = Math.max(state.learnedAt, at);
|
|
109
|
-
const d = learning.halfLife === null ? 1 : decayFactor(at - state.learnedAt, learning.halfLife);
|
|
110
|
-
if (d === 1) return learnedAt === state.learnedAt ? state : { ...state, learnedAt };
|
|
111
|
-
return {
|
|
112
|
-
...state,
|
|
113
|
-
learnedAt,
|
|
114
|
-
learned: mapRecord(state.learned, (r) => ({ hits: r.hits * d, misses: r.misses * d })),
|
|
115
|
-
episodes: { hits: state.episodes.hits * d, misses: state.episodes.misses * d }
|
|
116
|
-
};
|
|
117
|
-
}
|
|
118
|
-
function signalWeights(state, intent, learning, signal) {
|
|
119
|
-
const initial = getOwn(intent.signals, signal);
|
|
120
|
-
if (initial === void 0) {
|
|
121
|
-
throw new RangeError(`oracular: ${JSON.stringify(signal)} is not a signal of this intent`);
|
|
122
|
-
}
|
|
123
|
-
const record2 = getOwn(state.learned, signal) ?? NONE;
|
|
124
|
-
const nWith = record2.hits + record2.misses;
|
|
125
|
-
const qWith = laplace(record2.hits, nWith);
|
|
126
|
-
const totalEpisodes = state.episodes.hits + state.episodes.misses;
|
|
127
|
-
const nWithout = Math.max(0, totalEpisodes - nWith);
|
|
128
|
-
const outcomesWithout = Math.min(nWithout, Math.max(0, state.episodes.hits - record2.hits));
|
|
129
|
-
const qWithout = laplace(outcomesWithout, nWithout);
|
|
130
|
-
const learned = nWith === 0 ? 0 : learning.gain * Math.abs(initial) * lift(qWith, qWithout);
|
|
131
|
-
const nEff = nWith + nWithout === 0 ? 0 : nWith * nWithout / (nWith + nWithout);
|
|
132
|
-
const k = learning.priorEpisodes;
|
|
133
|
-
const effective = (k * initial + nEff * learned) / (k + nEff);
|
|
134
|
-
return { initial, learned, effective, hits: record2.hits, misses: record2.misses };
|
|
135
|
-
}
|
|
136
|
-
function recordEpisode(state, signals, outcome) {
|
|
137
|
-
const learned = { ...state.learned };
|
|
138
|
-
for (const signal of signals) {
|
|
139
|
-
const r = getOwn(learned, signal) ?? NONE;
|
|
140
|
-
learned[signal] = outcome ? { hits: r.hits + 1, misses: r.misses } : { hits: r.hits, misses: r.misses + 1 };
|
|
141
|
-
}
|
|
142
|
-
const episodes = outcome ? { hits: state.episodes.hits + 1, misses: state.episodes.misses } : { hits: state.episodes.hits, misses: state.episodes.misses + 1 };
|
|
143
|
-
return { ...state, learned, episodes };
|
|
144
|
-
}
|
|
145
|
-
|
|
146
185
|
// src/config.ts
|
|
147
186
|
var DEFAULT_THRESHOLD = 0.5;
|
|
148
187
|
var DEFAULT_Z = 1.28;
|
|
@@ -158,6 +197,7 @@ var INTENT_KEYS = [
|
|
|
158
197
|
"outcome",
|
|
159
198
|
"resetOnOutcome",
|
|
160
199
|
"saturate",
|
|
200
|
+
"attribution",
|
|
161
201
|
"reveal"
|
|
162
202
|
];
|
|
163
203
|
var REVEAL_KEYS = ["threshold", "z", "cooldown"];
|
|
@@ -287,6 +327,9 @@ function normalizeIntentConfig(value, path) {
|
|
|
287
327
|
if (typeof record2.saturate !== "boolean") typeError(`${path}.saturate`, "must be a boolean");
|
|
288
328
|
out.saturate = record2.saturate;
|
|
289
329
|
}
|
|
330
|
+
if (record2.attribution !== void 0) {
|
|
331
|
+
out.attribution = expectPositive(record2.attribution, `${path}.attribution`);
|
|
332
|
+
}
|
|
290
333
|
if (record2.reveal !== void 0)
|
|
291
334
|
out.reveal = normalizeRevealOptions(record2.reveal, `${path}.reveal`);
|
|
292
335
|
return out;
|
|
@@ -409,7 +452,7 @@ function markResolved(episode, intent) {
|
|
|
409
452
|
}
|
|
410
453
|
|
|
411
454
|
// src/version.ts
|
|
412
|
-
var STATE_VERSION =
|
|
455
|
+
var STATE_VERSION = 3;
|
|
413
456
|
var RECORD_VERSION = 1;
|
|
414
457
|
|
|
415
458
|
// src/state.ts
|
|
@@ -440,7 +483,8 @@ function getIntentState(state, name, intent) {
|
|
|
440
483
|
|
|
441
484
|
// src/assess.ts
|
|
442
485
|
function assessIntent(state, name, intent, learning, at) {
|
|
443
|
-
|
|
486
|
+
const is = decayIntent(getIntentState(state, name, intent), intent, at);
|
|
487
|
+
return settlePending(is, intent, learning, at);
|
|
444
488
|
}
|
|
445
489
|
function cooldownEnd(since, cooldown, at) {
|
|
446
490
|
if (since === null) return null;
|
|
@@ -513,6 +557,14 @@ function evaluate(state, intent, at, opts) {
|
|
|
513
557
|
explanation: buildExplanation(is, config, learning, split.explain)
|
|
514
558
|
};
|
|
515
559
|
}
|
|
560
|
+
function inspect(state, at, opts) {
|
|
561
|
+
const s = requireState(state);
|
|
562
|
+
const intents = Object.keys(s.config.intents).map((intent) => ({
|
|
563
|
+
intent,
|
|
564
|
+
...evaluate(s, intent, at, opts)
|
|
565
|
+
}));
|
|
566
|
+
return { at, intents };
|
|
567
|
+
}
|
|
516
568
|
|
|
517
569
|
// src/evidence.ts
|
|
518
570
|
function evidenceToReveal(intent, opts) {
|
|
@@ -567,7 +619,7 @@ function observe(state, event) {
|
|
|
567
619
|
if (!isSignal && !isOutcome) continue;
|
|
568
620
|
let is = getIntentState(s, intentName, intent);
|
|
569
621
|
is = decayIntent(is, intent, at);
|
|
570
|
-
is =
|
|
622
|
+
is = settlePending(is, intent, learning, at);
|
|
571
623
|
if (isSignal) {
|
|
572
624
|
const { effective } = signalWeights(is, intent, learning, name);
|
|
573
625
|
const n = occurrences(episode, intentName, name) + 1;
|
|
@@ -577,7 +629,8 @@ function observe(state, event) {
|
|
|
577
629
|
if (!isOutcome) episode = markSeen(episode, intentName, name);
|
|
578
630
|
}
|
|
579
631
|
if (isOutcome) {
|
|
580
|
-
is = recordEpisode(is, episodeRecord(episode, intentName).seen,
|
|
632
|
+
is = recordEpisode(is, episodeRecord(episode, intentName).seen, 1);
|
|
633
|
+
is = creditPending(is, intent, at);
|
|
581
634
|
episode = markResolved(episode, intentName);
|
|
582
635
|
is = { ...is, lastOutcomeAt: at };
|
|
583
636
|
if (intent.resetOnOutcome ?? true) is = resetToPrior(is, intent);
|
|
@@ -600,8 +653,8 @@ function closeEpisode(state, at) {
|
|
|
600
653
|
const record2 = episodeRecord(state.episode, intentName);
|
|
601
654
|
if (record2.resolved) continue;
|
|
602
655
|
let is = getIntentState(state, intentName, intent);
|
|
603
|
-
is =
|
|
604
|
-
is = recordEpisode(is, record2.seen,
|
|
656
|
+
is = settlePending(is, intent, learning, at);
|
|
657
|
+
is = intent.attribution === void 0 ? recordEpisode(is, record2.seen, 0) : deferEpisode(is, record2.seen, at);
|
|
605
658
|
intents[intentName] = is;
|
|
606
659
|
}
|
|
607
660
|
return { ...state, intents, episode: emptyEpisode() };
|
|
@@ -709,7 +762,8 @@ function createEndRecord(at, opts) {
|
|
|
709
762
|
|
|
710
763
|
// src/migrate.ts
|
|
711
764
|
var MIGRATIONS = {
|
|
712
|
-
1: fromV1
|
|
765
|
+
1: fromV1,
|
|
766
|
+
2: fromV2
|
|
713
767
|
};
|
|
714
768
|
function canMigrate(version) {
|
|
715
769
|
return hasOwn(MIGRATIONS, String(version));
|
|
@@ -740,6 +794,16 @@ function fromV1(stored) {
|
|
|
740
794
|
}
|
|
741
795
|
return { ...stored, version: 2, episode: { ...episode, intents } };
|
|
742
796
|
}
|
|
797
|
+
function fromV2(stored) {
|
|
798
|
+
if (!isRecord(stored.intents)) return { ...stored, version: 3 };
|
|
799
|
+
const intents = {};
|
|
800
|
+
for (const key of Object.keys(stored.intents)) {
|
|
801
|
+
if (key === "__proto__") continue;
|
|
802
|
+
const intent = stored.intents[key];
|
|
803
|
+
intents[key] = isRecord(intent) ? { ...intent, pending: [] } : intent;
|
|
804
|
+
}
|
|
805
|
+
return { ...stored, version: 3, intents };
|
|
806
|
+
}
|
|
743
807
|
|
|
744
808
|
// src/serialize.ts
|
|
745
809
|
function serialize(state) {
|
|
@@ -771,9 +835,11 @@ function deserialize(json, opts = {}) {
|
|
|
771
835
|
if (parsed.version !== STATE_VERSION) {
|
|
772
836
|
if (canMigrate(parsed.version)) {
|
|
773
837
|
stored = migrateStored(parsed);
|
|
838
|
+
} else if (parsed.version === STATE_VERSION + 1) {
|
|
839
|
+
stored = { ...parsed, version: STATE_VERSION };
|
|
774
840
|
} else if (opts.migrate === void 0) {
|
|
775
841
|
throw new RangeError(
|
|
776
|
-
`oracular: stored state is version ${parsed.version} but this library reads versions 1 to ${STATE_VERSION}; pass a migrate function`
|
|
842
|
+
`oracular: stored state is version ${parsed.version} but this library reads versions 1 to ${STATE_VERSION + 1}; pass a migrate function`
|
|
777
843
|
);
|
|
778
844
|
} else {
|
|
779
845
|
stored = opts.migrate(parsed, parsed.version);
|
|
@@ -826,6 +892,19 @@ function mapOwn(value, path, fn) {
|
|
|
826
892
|
}
|
|
827
893
|
return out;
|
|
828
894
|
}
|
|
895
|
+
function pendingEpisodes(value, path) {
|
|
896
|
+
if (!Array.isArray(value)) typeError2(path, "must be an array");
|
|
897
|
+
if (value.length > MAX_PENDING) {
|
|
898
|
+
throw new RangeError(`oracular: ${path} must hold at most ${MAX_PENDING} episodes`);
|
|
899
|
+
}
|
|
900
|
+
return value.map((item, i) => {
|
|
901
|
+
const r = record(item, `${path}[${i}]`);
|
|
902
|
+
if (!Array.isArray(r.seen) || !r.seen.every((s) => typeof s === "string")) {
|
|
903
|
+
typeError2(`${path}[${i}].seen`, "must be an array of strings");
|
|
904
|
+
}
|
|
905
|
+
return { endedAt: finite(r.endedAt, `${path}[${i}].endedAt`), seen: r.seen };
|
|
906
|
+
});
|
|
907
|
+
}
|
|
829
908
|
function intentState(value, path) {
|
|
830
909
|
const r = record(value, path);
|
|
831
910
|
const revealCount = finite(r.revealCount, `${path}.revealCount`);
|
|
@@ -839,6 +918,7 @@ function intentState(value, path) {
|
|
|
839
918
|
contributions: mapOwn(r.contributions, `${path}.contributions`, contribution),
|
|
840
919
|
learned: mapOwn(r.learned, `${path}.learned`, signalRecord),
|
|
841
920
|
episodes: signalRecord(r.episodes, `${path}.episodes`),
|
|
921
|
+
pending: pendingEpisodes(r.pending, `${path}.pending`),
|
|
842
922
|
learnedAt: nullOrFinite(r.learnedAt, `${path}.learnedAt`),
|
|
843
923
|
lastRevealedAt: nullOrFinite(r.lastRevealedAt, `${path}.lastRevealedAt`),
|
|
844
924
|
lastOutcomeAt: nullOrFinite(r.lastOutcomeAt, `${path}.lastOutcomeAt`),
|
|
@@ -907,6 +987,12 @@ function keep(source, declared) {
|
|
|
907
987
|
}
|
|
908
988
|
return out;
|
|
909
989
|
}
|
|
990
|
+
function keepPending(pending, signals) {
|
|
991
|
+
return pending.map((episode) => ({
|
|
992
|
+
endedAt: episode.endedAt,
|
|
993
|
+
seen: episode.seen.filter((s, i) => hasOwn(signals, s) && episode.seen.indexOf(s) === i)
|
|
994
|
+
})).sort((a, b) => a.endedAt - b.endedAt);
|
|
995
|
+
}
|
|
910
996
|
function reconcile(raw, config) {
|
|
911
997
|
const intents = {};
|
|
912
998
|
const episodeIntents = {};
|
|
@@ -916,7 +1002,11 @@ function reconcile(raw, config) {
|
|
|
916
1002
|
intents[name] = initIntent(intent);
|
|
917
1003
|
} else {
|
|
918
1004
|
intents[name] = withContributions(
|
|
919
|
-
{
|
|
1005
|
+
{
|
|
1006
|
+
...stored,
|
|
1007
|
+
learned: keep(stored.learned, intent.signals),
|
|
1008
|
+
pending: keepPending(stored.pending, intent.signals)
|
|
1009
|
+
},
|
|
920
1010
|
intent,
|
|
921
1011
|
keep(stored.contributions, intent.signals)
|
|
922
1012
|
);
|
|
@@ -968,6 +1058,7 @@ exports.episodeClosed = episodeClosed;
|
|
|
968
1058
|
exports.evaluate = evaluate;
|
|
969
1059
|
exports.evidenceToReveal = evidenceToReveal;
|
|
970
1060
|
exports.explain = explain;
|
|
1061
|
+
exports.inspect = inspect;
|
|
971
1062
|
exports.markRevealed = markRevealed;
|
|
972
1063
|
exports.ms = ms;
|
|
973
1064
|
exports.observe = observe;
|