oracular 0.14.0 → 0.16.0

package/CHANGELOG.md

@@ -1,5 +1,19 @@
 # Changelog
 
+## 0.16.0
+
+### Minor Changes
+
+- [#74](https://github.com/Yuribenjamin/oracular/pull/74) [`7e6b6fc`](https://github.com/Yuribenjamin/oracular/commit/7e6b6fc3885dac0167b92661e188e3f779a73a45) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **The outcome record keeps the estimate the reset wipes.** `createRecord` takes the state after `observe`, so on the record of an intent's own outcome event `p` is what the engine decides right after it, with `resetOnOutcome` the prior (0.150 where it had been 0.506 one event earlier). Replay recomputes everything from the events and never reads it, but people read the log, and the number that matters most on an outcome line was gone. `createRecord(state, event, options, previous)` takes the state from before the event as an optional fourth argument; the decision of each intent whose outcome the event is then gains `before`, its `p` and `lower` at `event.at` on that state. Every other record is byte for byte what it was. `RECORD_VERSION` stays 1: a new optional field does not change it, readers ignore fields they do not know, and `replay` and `tune` never read decisions, so a log with `before` replays on every release that reads version 1. Found by the second production integration.
+
+## 0.15.0
+
+### Minor Changes
+
+- [#73](https://github.com/Yuribenjamin/oracular/pull/73) [`3363311`](https://github.com/Yuribenjamin/oracular/commit/3363311a66fa2f3208a3be43f71523ac93979101) Thanks [@Yuribenjamin](https://github.com/Yuribenjamin)! - **An episode per outcome, for intents shaped as attempts.** The episode is the session, shared by every intent, so for an upload that fails, completes, and completes again in one session the engine counted one episode: the second completion, with nothing seen before it, was a duplicate under 0.10's rule, and a second attempt that failed and never completed stayed neutral at the close. The failure signal never saw an outcome without it, which is the contrast learning needs, and `tune` printed "1 counted episode" where the app saw two. `IntentConfig.episodePerOutcome: true` makes each outcome an episode of its own for that intent: a later outcome in the same session counts as another episode even with nothing seen since, and signals seen after an outcome take a miss at the session's close, or wait under `attribution`. Everything else is as before, `resetOnOutcome` is independent, and `tune` follows the engine's counts. Off unless set: every state and every number without it is exactly what it was. With it set, a duplicated outcome event counts twice, so dedupe such events in the app for these intents. The JSON Schema lists it. Found by the second production integration, whose workaround, `endEpisode` on the outcome, ends the session for every other intent too.
+  
+  **Rollback.** A stored config that sets `episodePerOutcome` loads in 0.13 and later, which do not read the stored snapshot when the app passes its config; 0.12 and earlier throw a `TypeError` on the unknown key, so remove it from the saved config before rolling a deploy back that far. The state version is unchanged.
+
 ## 0.14.0
 
 ### Minor Changes

package/README.md

@@ -186,7 +186,7 @@ Before any update or reading, the evidence decays by 2^(−Δt / halfLife), wher
 
 ### The user's history corrects your weights
 
-Events happen inside **episodes** (sessions). An episode ends when you call `endEpisode`, or on its own after `episodeTimeout` of inactivity. Signals seen in the episode are remembered per intent. When the intent's `outcome` event arrives, each remembered signal gets a **hit**; when the episode ends without it, each gets a **miss**, or, with `attribution` set on the intent, the episode waits and takes the share `2^(−age / halfLife)` of a hit if the outcome arrives inside the window. The engine also counts, per intent, how many episodes ended with and without the outcome: the intent's **base rate**. Signals seen after an outcome in the same episode count only if another outcome follows them; a repeated outcome with nothing seen since counts nothing.
+Events happen inside **episodes** (sessions). An episode ends when you call `endEpisode`, or on its own after `episodeTimeout` of inactivity. Signals seen in the episode are remembered per intent. When the intent's `outcome` event arrives, each remembered signal gets a **hit**; when the episode ends without it, each gets a **miss**, or, with `attribution` set on the intent, the episode waits and takes the share `2^(−age / halfLife)` of a hit if the outcome arrives inside the window. The engine also counts, per intent, how many episodes ended with and without the outcome: the intent's **base rate**. Signals seen after an outcome in the same episode count only if another outcome follows them; a repeated outcome with nothing seen since counts nothing. For an intent shaped as attempts (an upload that fails, completes, and completes again), set `episodePerOutcome` and each outcome is an episode of its own: the second completion counts, with nothing seen, as the contrast learning needs, and failures after a completion take a miss when the session ends.
 
 A signal's weight is then a blend of your prior and the learned estimate:
 
@@ -264,7 +264,7 @@ evaluate(state, intent: string, at: number, opts?): { prediction, reveal, explan
 inspect(state, at: number, opts?): { at, intents: [{ intent, prediction, reveal, explanation }] }  // evaluate for every intent
 evidenceToReveal(intentConfig, opts?): number   // pseudo-counts at which shouldReveal first becomes true
 episodeClosed(prev, next): boolean               // did an episode close between two states?
-createRecord(state, event, { installId, mode }): EventRecord   // one recorder line: the event and what was decided
+createRecord(state, event, { installId, mode }, previous?): EventRecord   // one recorder line: the event and what was decided; with the state before the event, the outcome's record keeps the estimate the reset wipes
 createEndRecord(at, { installId, mode }): EndRecord           // one recorder line: the episode ended
 serialize(state): string
 deserialize(json: string, opts?: { config?: EngineConfig; migrate?: Migrate }): EngineState
@@ -297,6 +297,7 @@ interface IntentConfig {
   outcome?: string;                 // event that confirms the intent
   resetOnOutcome?: boolean;         // default true
   saturate?: boolean;               // repeats within an episode add less; default: episodeTimeout is set
+  episodePerOutcome?: boolean;      // each outcome is an episode of its own for this intent; default false: the session is
   attribution?: number;             // how long a closed episode waits for the outcome before it is a miss; off if omitted
   reveal?: RevealOptions;
 }

package/dist/index.cjs

@@ -286,6 +286,7 @@ var INTENT_KEYS = [
   "outcome",
   "resetOnOutcome",
   "saturate",
+  "episodePerOutcome",
   "attribution",
   "reveal"
 ];
@@ -443,6 +444,12 @@ function normalizeIntentConfig(value, path, strict = true) {
     if (typeof record2.saturate !== "boolean") typeError(`${path}.saturate`, "must be a boolean");
     out.saturate = record2.saturate;
   }
+  if (record2.episodePerOutcome !== void 0) {
+    if (typeof record2.episodePerOutcome !== "boolean") {
+      typeError(`${path}.episodePerOutcome`, "must be a boolean");
+    }
+    out.episodePerOutcome = record2.episodePerOutcome;
+  }
   if (record2.attribution !== void 0) {
     out.attribution = expectPositive(record2.attribution, `${path}.attribution`);
   }
@@ -759,7 +766,9 @@ function observe(state, event) {
     }
     if (isOutcome) {
       const record2 = episodeRecord(episode, intentName);
-      if (!record2.resolved || record2.seen.length > 0) is = recordEpisode(is, record2.seen, 1);
+      if (!record2.resolved || record2.seen.length > 0 || intent.episodePerOutcome === true) {
+        is = recordEpisode(is, record2.seen, 1);
+      }
       is = creditPending(is, intent, at);
       episode = markResolved(episode, intentName);
       is = { ...is, lastOutcomeAt: at };
@@ -812,7 +821,8 @@ function closeEpisode(state, at) {
   const intents = { ...state.intents };
   for (const [intentName, intent] of Object.entries(state.config.intents)) {
     const record2 = episodeRecord(state.episode, intentName);
-    if (record2.resolved) continue;
+    if (record2.resolved && (intent.episodePerOutcome !== true || record2.seen.length === 0))
+      continue;
     let is = getIntentState(state, intentName, intent);
     is = settlePending(is, intent, learning, at);
     is = intent.attribution === void 0 ? recordEpisode(is, record2.seen, 0) : deferEpisode(is, record2.seen, at);
@@ -889,10 +899,11 @@ function normalizeRecordOptions(value) {
   }
   return { installId, mode, top };
 }
-function createRecord(state, event, opts) {
+function createRecord(state, event, opts, previous) {
   const s = requireState(state);
   const { name, at } = validateEvent(event);
   const { installId, mode, top } = normalizeRecordOptions(opts);
+  const prev = previous === void 0 ? void 0 : requireState(previous);
   const learning = resolveLearning(s.config);
   const decisions = {};
   for (const [intentName, intent] of Object.entries(s.config.intents)) {
@@ -904,7 +915,7 @@ function createRecord(state, event, opts) {
     for (const signal of Object.keys(intent.signals)) {
       weights[signal] = rows.find((row) => row.signal === signal).effectiveWeight;
     }
-    decisions[intentName] = {
+    const decision = {
       p: prediction.p,
       lower: prediction.lower,
       shouldReveal: decideReveal(prediction, reveal),
@@ -912,6 +923,17 @@ function createRecord(state, event, opts) {
       top: rows.filter((row) => row.contribution !== 0).slice(0, top).map((row) => ({ signal: row.signal, impact: row.impact })),
       weights
     };
+    const prevIntent = prev === void 0 ? void 0 : getOwn(prev.config.intents, intentName);
+    if (prev !== void 0 && prevIntent !== void 0 && intent.outcome === name) {
+      const was = buildPrediction(
+        assessIntent(prev, intentName, prevIntent, resolveLearning(prev.config), at),
+        intentName,
+        resolveReveal(prev.config, prevIntent, void 0),
+        at
+      );
+      decision.before = { p: was.p, lower: was.lower };
+    }
+    decisions[intentName] = decision;
   }
   return { v: RECORD_VERSION, kind: "event", installId, mode, event: { name, at }, decisions };
 }