@zukhruf/relevance-filtering 0.6.2 → 0.6.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -20,30 +20,41 @@ npm install @zukhruf/relevance-filtering ai
20
20
 
21
21
  ### With a decision model
22
22
 
23
- Give the scorer an AI SDK decision model, for example Liquid AI's d1-2B on a llama.cpp server through `@ai-sdk/typesafe-ai`, and the description of the interest.
23
+ Give the scorer an AI SDK decision model, for example Liquid AI's d1-2B on a llama.cpp server through `@ai-sdk/typesafe-ai`, and the description of the interest. Wrap the model once in a `CoalescingDecisionModel`, and score the interests together with `Promise.all`. Then the model gets one call for each article with the questions of all the interests.
24
24
 
25
25
  ```ts
26
26
  import { createTypeSafeAi } from '@ai-sdk/typesafe-ai';
27
27
 
28
- import { DecisionScorer, UtilityThreshold } from '@zukhruf/relevance-filtering';
28
+ import {
29
+ CoalescingDecisionModel,
30
+ DecisionScorer,
31
+ UtilityThreshold,
32
+ } from '@zukhruf/relevance-filtering';
29
33
 
30
- const scorer = new DecisionScorer({
31
- model: createTypeSafeAi({
34
+ const model = new CoalescingDecisionModel(
35
+ createTypeSafeAi({
32
36
  baseURL: 'http://127.0.0.1:9931/v1',
33
37
  apiKey: 'none',
34
38
  }).decisionModel('LiquidAI/d1-2B-GGUF:Q8_0'),
35
- description: 'Cybersecurity: attacks, breaches and the tools that stop them.',
36
- });
39
+ );
40
+ const descriptions = [
41
+ 'Cybersecurity: attacks, breaches and the tools that stop them.',
42
+ 'New computer chips and AI hardware, and how chips are made and supplied.',
43
+ ];
37
44
 
38
45
  // The new articles by id: { publishedAt, text: `${title} ${lead}` }. The
39
46
  // signal stops the questions to the model, for example when the host stops.
40
- const scores = await scorer.score(newArticles, { signal });
47
+ const [security, chips] = await Promise.all(
48
+ descriptions.map((description) =>
49
+ new DecisionScorer({ model, description }).score(newArticles, { signal }),
50
+ ),
51
+ );
41
52
 
42
53
  // The best 5 articles of one day, the best first.
43
- scores.ranked({ from: dayStart, to: dayEnd }).slice(0, 5);
44
- scores.probability(id); // from 0 to 1
54
+ security.ranked({ from: dayStart, to: dayEnd }).slice(0, 5);
55
+ security.probability(id); // from 0 to 1
45
56
  // Notify only when a miss costs three times as much as a false alarm.
46
- scores.accepted(new UtilityThreshold({ falseAlarm: 1, miss: 3 }));
57
+ security.accepted(new UtilityThreshold({ falseAlarm: 1, miss: 3 }));
47
58
  ```
48
59
 
49
60
  ### With embeddings
@@ -108,7 +119,8 @@ scores.ranked({ from: dayStart, to: dayEnd }).slice(0, 5);
108
119
 
109
120
  - A `Scorer` has `score(stream, { signal })`, which gives a `Promise` of `Scores`. A scorer is made for one interest ([ADR 0006](./docs/adr/0006-a-scorer-is-made-for-one-interest-and-its-inputs-rule-out-illegal-states.md)). The `stream` is a map from article id to `{ publishedAt, text }`; `text` is, for example, the title and the lead. It can be empty.
110
121
  - The `signal` is an `AbortSignal`, and it is required ([ADR 0007](./docs/adr/0007-a-score-takes-a-signal.md)). When it aborts, the score rejects with the reason of the signal. Give the signal of the request or of the host's stop. A host that cannot stop a score gives `new AbortController().signal`.
111
- - `new DecisionScorer({ model, description })` takes an AI SDK decision model and the description. For each article, it asks the model "Is the main subject of this article: <description>?" about `text`, one article at a time. It gives the signal to the AI SDK, so a stop ends the question that runs, and the scorer asks about no later article.
122
+ - `new DecisionScorer({ model, description })` takes an AI SDK decision model and the description. For each article, it asks the model "Is the main subject of this article: <description>?" about `text`. It keeps up to 8 calls in flight, two times the 4 slots that a llama.cpp server makes by default ([ADR 0008](./docs/adr/0008-a-decision-scorer-keeps-eight-calls-in-flight-and-a-coalescing-model-asks-one-article-once.md)). It gives the signal to each call, so a stop aborts the calls that run, and no question starts after the stop. When a call fails, the scorer aborts the other calls. The ranking does not depend on the order in which the answers arrive.
123
+ - `new CoalescingDecisionModel(model)` wraps an AI SDK decision model, and is a decision model too. The calls that reach it before the next macrotask, about the same text and with equal options, go to the wrapped model as one call with all their questions (request coalescing, as in DataLoader). Each caller gets only its own answers. Wrap the model once, make one `DecisionScorer` for each interest with it, and score the interests with `Promise.all`. A call that is alone, has a state with a part that is not text, or has other options goes to the wrapped model without a change. A caller that aborts rejects at once; the merged call aborts only when all its callers abort. An error of the wrapped model rejects each caller of that call, and a refused question fails only its caller. Only the first caller of a merged call gets the usage, so a sum over the callers is the real usage.
112
124
  - `new SimilarityScorer({ vectors, descriptionVector, answers, keywords })` is the frozen design for an interest with at least one yes answer.
113
125
  - `vectors` is the port to the vectors of the articles: `query(vector, { from, to })` and `fetch(ids)`. The `Collection` of `@zukhruf/embeddings` has this shape. The stream and the taught articles must be in it, and `publishedAt` must be the date that the vectors store for the article.
114
126
  - `descriptionVector` is the vector of the description from the model of the stored vectors, made with the prompt that the model uses for a query.
@@ -165,7 +177,7 @@ A keyword list puts the matches first, the newest first. The 95% interval of (si
165
177
  ## Limits
166
178
 
167
179
  - The decision scorer was measured only with d1-2B (Q8_0), the question of round 13, and the title and the lead as the text. Another model or another question needs a new research round.
168
- - The decision scorer calls the model once for each article of the stream. The host decides when to score and if it keeps the scores.
180
+ - The decision scorer calls the model once for each article of the stream. With a `CoalescingDecisionModel`, the scorers of all interests that score together share one call for each article. The host decides when to score and if it keeps the scores.
169
181
  - The answer keys of rounds 13 and 14 come from language models. No person labelled that slice (limitation 16 of the research record).
170
182
  - The similarity design is valid only with EmbeddingGemma 2, the vector text of event detection (`title: <title> | text: <title>\n<lead>`) for the articles and the query prompt for the description. Another model gives another similarity scale.
171
183
  - The binormal model is fitted on the stream. A stream of a few articles gives a weak fit. The research streams held 1,600 to 1,700 articles.
@@ -184,6 +196,8 @@ Before you change a step of the similarity design, read ADR 0002, and run `nx ru
184
196
  - [ADR 0004: The reader ranks, and does not decide yes or no](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md)
185
197
  - [ADR 0005: A scorer and a threshold are ports](./docs/adr/0005-a-scorer-and-a-threshold-are-ports.md)
186
198
  - [ADR 0006: A scorer is made for one interest, and its inputs rule out illegal states](./docs/adr/0006-a-scorer-is-made-for-one-interest-and-its-inputs-rule-out-illegal-states.md)
199
+ - [ADR 0007: A score takes a signal](./docs/adr/0007-a-score-takes-a-signal.md)
200
+ - [ADR 0008: A decision scorer keeps eight calls in flight, and a coalescing model asks about one article once](./docs/adr/0008-a-decision-scorer-keeps-eight-calls-in-flight-and-a-coalescing-model-asks-one-article-once.md)
187
201
  - [Code copied from the event-detection eval](./docs/copied-code.md)
188
202
 
189
203
  ## Recipes
@@ -0,0 +1,17 @@
1
+ import type { Experimental_DecisionModel } from 'ai';
2
+ type DecisionModel = Extract<Experimental_DecisionModel, {
3
+ doDecide: unknown;
4
+ }>;
5
+ type DecideOptions = Parameters<DecisionModel['doDecide']>[0];
6
+ type DecideResult = Awaited<ReturnType<DecisionModel['doDecide']>>;
7
+ export declare class CoalescingDecisionModel implements DecisionModel {
8
+ #private;
9
+ readonly specificationVersion = "v4";
10
+ readonly provider: string;
11
+ readonly modelId: string;
12
+ readonly supportedQuestionTypes: DecisionModel['supportedQuestionTypes'];
13
+ constructor(inner: DecisionModel);
14
+ doDecide(options: DecideOptions): PromiseLike<DecideResult>;
15
+ }
16
+ export {};
17
+ //# sourceMappingURL=coalescing-decision-model.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"coalescing-decision-model.d.ts","sourceRoot":"","sources":["../src/coalescing-decision-model.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,0BAA0B,EAAE,MAAM,IAAI,CAAC;AAIrD,KAAK,aAAa,GAAG,OAAO,CAAC,0BAA0B,EAAE;IAAE,QAAQ,EAAE,OAAO,CAAA;CAAE,CAAC,CAAC;AAChF,KAAK,aAAa,GAAG,UAAU,CAAC,aAAa,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;AAC9D,KAAK,YAAY,GAAG,OAAO,CAAC,UAAU,CAAC,aAAa,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC;AAgCnE,qBAAa,uBAAwB,YAAW,aAAa;;IAC3D,QAAQ,CAAC,oBAAoB,QAAQ;IACrC,QAAQ,CAAC,QAAQ,EAAE,MAAM,CAAC;IAC1B,QAAQ,CAAC,OAAO,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,sBAAsB,EAAE,aAAa,CAAC,wBAAwB,CAAC,CAAC;gBAK7D,KAAK,EAAE,aAAa;IAOhC,QAAQ,CAAC,OAAO,EAAE,aAAa,GAAG,WAAW,CAAC,YAAY,CAAC;CA6G5D"}
@@ -0,0 +1,125 @@
1
+ const aborted = ({ options }) => options.abortSignal?.aborted === true;
2
+ // Request coalescing, the pattern of Facebook's DataLoader: the calls that
3
+ // reach the model in one turn of the event loop, about the same text, go to
4
+ // the inner model as one call with all their questions. A decision model such
5
+ // as TypeSafe System One answers all the questions of a call in one pass over
6
+ // the state, so the scorers of several interests ask about one article for
7
+ // the cost of about one question (docs/adr/0008).
8
+ //
9
+ // The window closes on setImmediate. experimental_decide of ai 7 awaits only
10
+ // promises that settle in microtasks before it calls doDecide (the onStart
11
+ // callbacks, the preparation of a text state, the retry wrapper), so the
12
+ // calls of several scorers that start together with Promise.all all reach
13
+ // doDecide before the next macrotask. A probe on ai 7.0.154 with four calls
14
+ // in Promise.all saw all four before the first setImmediate.
15
+ //
16
+ // A call goes to the inner model unchanged, with its own signal, when it is
17
+ // alone in its window, when its state holds a part that is not text, or when
18
+ // its other options differ from those of every other call.
19
+ export class CoalescingDecisionModel {
20
+ specificationVersion = 'v4';
21
+ provider;
22
+ modelId;
23
+ supportedQuestionTypes;
24
+ #inner;
25
+ // The open window: the calls that wait, by their state and options.
26
+ #window = new Map();
27
+ constructor(inner) {
28
+ this.#inner = inner;
29
+ this.provider = inner.provider;
30
+ this.modelId = inner.modelId;
31
+ this.supportedQuestionTypes = inner.supportedQuestionTypes;
32
+ }
33
+ doDecide(options) {
34
+ if (options.state.some((part) => part.type !== 'text'))
35
+ return this.#inner.doDecide(options);
36
+ const { state, maxOutputTokens, reasoning, headers, providerOptions } = options;
37
+ const key = JSON.stringify({
38
+ state,
39
+ maxOutputTokens,
40
+ reasoning,
41
+ headers,
42
+ providerOptions,
43
+ });
44
+ return new Promise((resolve, reject) => {
45
+ const { abortSignal } = options;
46
+ abortSignal?.throwIfAborted();
47
+ const group = this.#group(key);
48
+ // Removes the abort listener when the call settles.
49
+ const listening = new AbortController();
50
+ const caller = {
51
+ options,
52
+ resolve: (result) => {
53
+ listening.abort();
54
+ resolve(result);
55
+ },
56
+ reject: (reason) => {
57
+ listening.abort();
58
+ reject(reason);
59
+ },
60
+ };
61
+ // A caller that aborts rejects at once with its reason.
62
+ abortSignal?.addEventListener('abort', () => {
63
+ caller.reject(abortSignal.reason);
64
+ if (group.callers.every(aborted))
65
+ group.abort.abort(abortSignal.reason);
66
+ }, { once: true, signal: listening.signal });
67
+ group.callers.push(caller);
68
+ });
69
+ }
70
+ // The group of the open window for the key. The first call of a window
71
+ // opens it, and the next macrotask sends its groups.
72
+ #group(key) {
73
+ if (this.#window.size === 0)
74
+ setImmediate(() => {
75
+ const groups = [...this.#window.values()];
76
+ this.#window.clear();
77
+ for (const group of groups)
78
+ this.#send(group);
79
+ });
80
+ const group = this.#window.get(key) ?? {
81
+ callers: [],
82
+ abort: new AbortController(),
83
+ };
84
+ this.#window.set(key, group);
85
+ return group;
86
+ }
87
+ #send(group) {
88
+ const callers = group.callers.filter((caller) => !aborted(caller));
89
+ const [first] = callers;
90
+ if (first === undefined)
91
+ return;
92
+ if (callers.length === 1) {
93
+ Promise.try(() => this.#inner.doDecide(first.options)).then(first.resolve, first.reject);
94
+ return;
95
+ }
96
+ // The ids of the merged call start with the index of their caller, so
97
+ // two callers' ids never collide.
98
+ const merged = (index, id) => `${index}.${id}`;
99
+ Promise.try(() => this.#inner.doDecide({
100
+ ...first.options,
101
+ abortSignal: group.abort.signal,
102
+ questions: Object.fromEntries(callers.flatMap(({ options }, index) => Object.entries(options.questions).map(([id, question]) => [
103
+ merged(index, id),
104
+ question,
105
+ ]))),
106
+ })).then(({ answers, usage, ...result }) => {
107
+ for (const [index, { options, resolve }] of callers.entries())
108
+ resolve({
109
+ ...result,
110
+ answers: Object.fromEntries(Object.keys(options.questions).flatMap((id) => {
111
+ const answer = answers[merged(index, id)];
112
+ return answer === undefined ? [] : [[id, answer]];
113
+ })),
114
+ // The inner model used these tokens once for all the callers.
115
+ // Only the first caller reports them, so a sum over the callers
116
+ // equals the real usage.
117
+ ...(index === 0 && { usage }),
118
+ });
119
+ }, (error) => {
120
+ for (const { reject } of callers)
121
+ reject(error);
122
+ });
123
+ }
124
+ }
125
+ //# sourceMappingURL=coalescing-decision-model.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"coalescing-decision-model.js","sourceRoot":"","sources":["../src/coalescing-decision-model.ts"],"names":[],"mappings":"AAmBA,MAAM,OAAO,GAAG,CAAC,EAAE,OAAO,EAAU,EAAE,EAAE,CAAC,OAAO,CAAC,WAAW,EAAE,OAAO,KAAK,IAAI,CAAC;AAE/E,2EAA2E;AAC3E,4EAA4E;AAC5E,8EAA8E;AAC9E,8EAA8E;AAC9E,2EAA2E;AAC3E,kDAAkD;AAClD,EAAE;AACF,6EAA6E;AAC7E,2EAA2E;AAC3E,yEAAyE;AACzE,0EAA0E;AAC1E,4EAA4E;AAC5E,6DAA6D;AAC7D,EAAE;AACF,4EAA4E;AAC5E,6EAA6E;AAC7E,2DAA2D;AAC3D,MAAM,OAAO,uBAAuB;IACzB,oBAAoB,GAAG,IAAI,CAAC;IAC5B,QAAQ,CAAS;IACjB,OAAO,CAAS;IAChB,sBAAsB,CAA0C;IAChE,MAAM,CAAgB;IAC/B,oEAAoE;IAC3D,OAAO,GAAG,IAAI,GAAG,EAAiB,CAAC;IAE5C,YAAY,KAAoB;QAC9B,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,QAAQ,GAAG,KAAK,CAAC,QAAQ,CAAC;QAC/B,IAAI,CAAC,OAAO,GAAG,KAAK,CAAC,OAAO,CAAC;QAC7B,IAAI,CAAC,sBAAsB,GAAG,KAAK,CAAC,sBAAsB,CAAC;IAC7D,CAAC;IAED,QAAQ,CAAC,OAAsB;QAC7B,IAAI,OAAO,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,IAAI,EAAE,EAAE,CAAC,IAAI,CAAC,IAAI,KAAK,MAAM,CAAC;YACpD,OAAO,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;QACvC,MAAM,EAAE,KAAK,EAAE,eAAe,EAAE,SAAS,EAAE,OAAO,EAAE,eAAe,EAAE,GACnE,OAAO,CAAC;QACV,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC;YACzB,KAAK;YACL,eAAe;YACf,SAAS;YACT,OAAO;YACP,eAAe;SAChB,CAAC,CAAC;QACH,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;YACrC,MAAM,EAAE,WAAW,EAAE,GAAG,OAAO,CAAC;YAChC,WAAW,EAAE,cAAc,EAAE,CAAC;YAC9B,MAAM,KAAK,GAAG,IAAI,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC;YAC/B,oDAAoD;YACpD,MAAM,SAAS,GAAG,IAAI,eAAe,EAAE,CAAC;YACxC,MAAM,MAAM,GAAW;gBACrB,OAAO;gBACP,OAAO,EAAE,CAAC,MAAM,EAAE,EAAE;oBAClB,SAAS,CAAC,KAAK,EAAE,CAAC;oBAClB,OAAO,CAAC,MAAM,CAAC,CAAC;gBAClB,CAAC;gBACD,MAAM,EAAE,CAAC,MAAM,EAAE,EAAE;oBACjB,SAAS,CAAC,KAAK,EAAE,CAAC;oBAClB,MAAM,CAAC,MAAM,CAAC,CAAC;gBACjB,CAAC;aACF,CAAC;YACF,wDAAwD;YACxD,WAAW,EAAE,gBAAgB,CAC3B,OAAO,EACP,GAAG,EAAE;gBACH,MAAM,CAAC,MAAM,CAAC,WAAW,CAAC,MAAM,CAAC,CAAC;gBAClC,IAAI,KAAK,CAAC,OAAO,CAAC,KAAK,CAAC,OAAO,CAAC;oBAC9B,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,WAAW,CAAC,MAAM,CAAC,CAAC;YAC1C,CAAC,EACD,EAAE,IAAI,EAAE,IAAI,EAAE,MAAM,EAAE,SAAS,CAAC,MAAM,EAAE,CACzC,CAAC;YACF,KAAK,CAAC,OAAO,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC;QAC7B,CAAC,CAAC,CAAC;IACL,CAAC;IAED,uEAAuE;IACvE,qDAAqD;IACrD,MAAM,CAAC,GAAW;QAChB,IAAI,IAAI,CAAC,OAAO,CAAC,IAAI,KAAK,CAAC;YACzB,YAAY,CAAC,GAAG,EAAE;gBAChB,MAAM,MAAM,GAAG,CAAC,GAAG,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;gBAC1C,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,CAAC;gBACrB,KAAK,MAAM,KAAK,IAAI,MAAM;oBAAE,IAAI,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;YAChD,CAAC,CAAC,CAAC;QACL,MAAM,KAAK,GAAG,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI;YACrC,OAAO,EAAE,EAAE;YACX,KAAK,EAAE,IAAI,eAAe,EAAE;SAC7B,CAAC;QACF,IAAI,CAAC,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC;QAC7B,OAAO,KAAK,CAAC;IACf,CAAC;IAED,KAAK,CAAC,KAAY;QAChB,MAAM,OAAO,GAAG,KAAK,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;QACnE,MAAM,CAAC,KAAK,CAAC,GAAG,OAAO,CAAC;QACxB,IAAI,KAAK,KAAK,SAAS;YAAE,OAAO;QAChC,IAAI,OAAO,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;YACzB,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC,OAAO,CAAC,CAAC,CAAC,IAAI,CACzD,KAAK,CAAC,OAAO,EACb,KAAK,CAAC,MAAM,CACb,CAAC;YACF,OAAO;QACT,CAAC;QACD,sEAAsE;QACtE,kCAAkC;QAClC,MAAM,MAAM,GAAG,CAAC,KAAa,EAAE,EAAU,EAAE,EAAE,CAAC,GAAG,KAAK,IAAI,EAAE,EAAE,CAAC;QAC/D,OAAO,CAAC,GAAG,CAAC,GAAG,EAAE,CACf,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC;YACnB,GAAG,KAAK,CAAC,OAAO;YAChB,WAAW,EAAE,KAAK,CAAC,KAAK,CAAC,MAAM;YAC/B,SAAS,EAAE,MAAM,CAAC,WAAW,CAC3B,OAAO,CAAC,OAAO,CAAC,CAAC,EAAE,OAAO,EAAE,EAAE,KAAK,EAAE,EAAE,CACrC,MAAM,CAAC,OAAO,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,QAAQ,CAAC,EAAE,EAAE,CAAC;gBACxD,MAAM,CAAC,KAAK,EAAE,EAAE,CAAC;gBACjB,QAAQ;aACT,CAAC,CACH,CACF;SACF,CAAC,CACH,CAAC,IAAI,CACJ,CAAC,EAAE,OAAO,EAAE,KAAK,EAAE,GAAG,MAAM,EAAE,EAAE,EAAE;YAChC,KAAK,MAAM,CAAC,KAAK,EAAE,EAAE,OAAO,EAAE,OAAO,EAAE,CAAC,IAAI,OAAO,CAAC,OAAO,EAAE;gBAC3D,OAAO,CAAC;oBACN,GAAG,MAAM;oBACT,OAAO,EAAE,MAAM,CAAC,WAAW,CACzB,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,SAAS,CAAC,CAAC,OAAO,CAAC,CAAC,EAAE,EAAE,EAAE;wBAC5C,MAAM,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,CAAC;wBAC1C,OAAO,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,EAAE,MAAM,CAAC,CAAC,CAAC;oBACpD,CAAC,CAAC,CACH;oBACD,8DAA8D;oBAC9D,gEAAgE;oBAChE,yBAAyB;oBACzB,GAAG,CAAC,KAAK,KAAK,CAAC,IAAI,EAAE,KAAK,EAAE,CAAC;iBAC9B,CAAC,CAAC;QACP,CAAC,EACD,CAAC,KAAc,EAAE,EAAE;YACjB,KAAK,MAAM,EAAE,MAAM,EAAE,IAAI,OAAO;gBAAE,MAAM,CAAC,KAAK,CAAC,CAAC;QAClD,CAAC,CACF,CAAC;IACJ,CAAC;CACF"}
@@ -1 +1 @@
1
- {"version":3,"file":"decision-scorer.d.ts","sourceRoot":"","sources":["../src/decision-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,0BAA0B,EAAuB,MAAM,IAAI,CAAC;AAE1E,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAClD,OAAO,EAAE,MAAM,EAAc,MAAM,aAAa,CAAC;AAOjD,qBAAa,cAAe,YAAW,MAAM;;gBAI/B,OAAO,EAAE;QACnB,QAAQ,CAAC,KAAK,EAAE,0BAA0B,CAAC;QAE3C,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;KAC9B;IAKK,KAAK,CACT,MAAM,EAAE,MAAM,EACd,EAAE,MAAM,EAAE,EAAE;QAAE,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAA;KAAE,GAC3C,OAAO,CAAC,MAAM,CAAC;CA4BnB"}
1
+ {"version":3,"file":"decision-scorer.d.ts","sourceRoot":"","sources":["../src/decision-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,0BAA0B,EAAuB,MAAM,IAAI,CAAC;AAE1E,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAClD,OAAO,EAAE,MAAM,EAAc,MAAM,aAAa,CAAC;AAajD,qBAAa,cAAe,YAAW,MAAM;;gBAI/B,OAAO,EAAE;QACnB,QAAQ,CAAC,KAAK,EAAE,0BAA0B,CAAC;QAE3C,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;KAC9B;IAKK,KAAK,CACT,MAAM,EAAE,MAAM,EACd,EAAE,MAAM,EAAE,EAAE;QAAE,QAAQ,CAAC,MAAM,EAAE,WAAW,CAAA;KAAE,GAC3C,OAAO,CAAC,MAAM,CAAC;CAkDnB"}
@@ -1,5 +1,10 @@
1
1
  import { experimental_decide } from 'ai';
2
2
  import { Scores, compareIds } from "./scores.js";
3
+ // The most calls to the model that one score keeps in flight. The server owns
4
+ // its capacity: llama.cpp gives itself 4 slots by default (--parallel -1, at
5
+ // b11960). Twice that keeps each slot fed while an answer travels back, so a
6
+ // slot does not wait on a round trip (docs/adr/0008).
7
+ const IN_FLIGHT = 8;
3
8
  // A decision model answers a typed question about a text with a calibrated
4
9
  // probability, for example Liquid AI's d1-2B. Research rounds 13 and 14
5
10
  // (#3383) found that its probability ranks better than the frozen
@@ -13,17 +18,39 @@ export class DecisionScorer {
13
18
  this.#instructions = `Is the main subject of this article: ${options.description}?`;
14
19
  }
15
20
  async score(stream, { signal }) {
16
- const scored = await Array.fromAsync(stream, async ([id, { publishedAt, text }]) => {
17
- const { answers } = await experimental_decide({
18
- model: this.#model,
19
- abortSignal: signal,
20
- state: text,
21
- questions: {
22
- relevant: { type: 'boolean', instructions: this.#instructions },
23
- },
24
- });
25
- return { id, publishedAt, probability: answers.relevant.probability };
26
- });
21
+ signal.throwIfAborted();
22
+ // The first failure stops the other calls, so a failed score asks the
23
+ // model no more. A stop of the host aborts the same signal.
24
+ const failed = new AbortController();
25
+ const abortSignal = AbortSignal.any([signal, failed.signal]);
26
+ // The workers share one iterator, so each article is asked once.
27
+ const articles = stream.entries();
28
+ const scored = [];
29
+ const ask = async () => {
30
+ try {
31
+ for (const [id, { publishedAt, text }] of articles) {
32
+ const { answers } = await experimental_decide({
33
+ model: this.#model,
34
+ abortSignal,
35
+ state: text,
36
+ questions: {
37
+ relevant: { type: 'boolean', instructions: this.#instructions },
38
+ },
39
+ });
40
+ scored.push({
41
+ id,
42
+ publishedAt,
43
+ probability: answers.relevant.probability,
44
+ });
45
+ }
46
+ }
47
+ catch (error) {
48
+ failed.abort(error);
49
+ throw error;
50
+ }
51
+ };
52
+ await Promise.all(Array.from({ length: IN_FLIGHT }, ask));
53
+ // The answers arrive in any order; the ranking does not depend on it.
27
54
  return new Scores(new Map(scored
28
55
  .toSorted((a, b) => b.probability - a.probability || compareIds(a.id, b.id))
29
56
  .map(({ id, publishedAt, probability }) => [
@@ -1 +1 @@
1
- {"version":3,"file":"decision-scorer.js","sourceRoot":"","sources":["../src/decision-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAmC,mBAAmB,EAAE,MAAM,IAAI,CAAC;AAG1E,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAEjD,2EAA2E;AAC3E,wEAAwE;AACxE,kEAAkE;AAClE,2EAA2E;AAC3E,8DAA8D;AAC9D,MAAM,OAAO,cAAc;IAChB,MAAM,CAA6B;IACnC,aAAa,CAAS;IAE/B,YAAY,OAIX;QACC,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;QAC5B,IAAI,CAAC,aAAa,GAAG,wCAAwC,OAAO,CAAC,WAAW,GAAG,CAAC;IACtF,CAAC;IAED,KAAK,CAAC,KAAK,CACT,MAAc,EACd,EAAE,MAAM,EAAoC;QAE5C,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,SAAS,CAClC,MAAM,EACN,KAAK,EAAE,CAAC,EAAE,EAAE,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC,EAAE,EAAE;YACpC,MAAM,EAAE,OAAO,EAAE,GAAG,MAAM,mBAAmB,CAAC;gBAC5C,KAAK,EAAE,IAAI,CAAC,MAAM;gBAClB,WAAW,EAAE,MAAM;gBACnB,KAAK,EAAE,IAAI;gBACX,SAAS,EAAE;oBACT,QAAQ,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,YAAY,EAAE,IAAI,CAAC,aAAa,EAAE;iBAChE;aACF,CAAC,CAAC;YACH,OAAO,EAAE,EAAE,EAAE,WAAW,EAAE,WAAW,EAAE,OAAO,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC;QACxE,CAAC,CACF,CAAC;QACF,OAAO,IAAI,MAAM,CACf,IAAI,GAAG,CACL,MAAM;aACH,QAAQ,CACP,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW,IAAI,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,CAClE;aACA,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,WAAW,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC;YACzC,EAAE;YACF,EAAE,WAAW,EAAE,WAAW,EAAE;SAC7B,CAAC,CACL,CACF,CAAC;IACJ,CAAC;CACF"}
1
+ {"version":3,"file":"decision-scorer.js","sourceRoot":"","sources":["../src/decision-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAmC,mBAAmB,EAAE,MAAM,IAAI,CAAC;AAG1E,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAEjD,8EAA8E;AAC9E,6EAA6E;AAC7E,6EAA6E;AAC7E,sDAAsD;AACtD,MAAM,SAAS,GAAG,CAAC,CAAC;AAEpB,2EAA2E;AAC3E,wEAAwE;AACxE,kEAAkE;AAClE,2EAA2E;AAC3E,8DAA8D;AAC9D,MAAM,OAAO,cAAc;IAChB,MAAM,CAA6B;IACnC,aAAa,CAAS;IAE/B,YAAY,OAIX;QACC,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;QAC5B,IAAI,CAAC,aAAa,GAAG,wCAAwC,OAAO,CAAC,WAAW,GAAG,CAAC;IACtF,CAAC;IAED,KAAK,CAAC,KAAK,CACT,MAAc,EACd,EAAE,MAAM,EAAoC;QAE5C,MAAM,CAAC,cAAc,EAAE,CAAC;QACxB,sEAAsE;QACtE,4DAA4D;QAC5D,MAAM,MAAM,GAAG,IAAI,eAAe,EAAE,CAAC;QACrC,MAAM,WAAW,GAAG,WAAW,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;QAC7D,iEAAiE;QACjE,MAAM,QAAQ,GAAG,MAAM,CAAC,OAAO,EAAE,CAAC;QAClC,MAAM,MAAM,GAIN,EAAE,CAAC;QACT,MAAM,GAAG,GAAG,KAAK,IAAI,EAAE;YACrB,IAAI,CAAC;gBACH,KAAK,MAAM,CAAC,EAAE,EAAE,EAAE,WAAW,EAAE,IAAI,EAAE,CAAC,IAAI,QAAQ,EAAE,CAAC;oBACnD,MAAM,EAAE,OAAO,EAAE,GAAG,MAAM,mBAAmB,CAAC;wBAC5C,KAAK,EAAE,IAAI,CAAC,MAAM;wBAClB,WAAW;wBACX,KAAK,EAAE,IAAI;wBACX,SAAS,EAAE;4BACT,QAAQ,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,YAAY,EAAE,IAAI,CAAC,aAAa,EAAE;yBAChE;qBACF,CAAC,CAAC;oBACH,MAAM,CAAC,IAAI,CAAC;wBACV,EAAE;wBACF,WAAW;wBACX,WAAW,EAAE,OAAO,CAAC,QAAQ,CAAC,WAAW;qBAC1C,CAAC,CAAC;gBACL,CAAC;YACH,CAAC;YAAC,OAAO,KAAK,EAAE,CAAC;gBACf,MAAM,CAAC,KAAK,CAAC,KAAK,CAAC,CAAC;gBACpB,MAAM,KAAK,CAAC;YACd,CAAC;QACH,CAAC,CAAC;QACF,MAAM,OAAO,CAAC,GAAG,CAAC,KAAK,CAAC,IAAI,CAAC,EAAE,MAAM,EAAE,SAAS,EAAE,EAAE,GAAG,CAAC,CAAC,CAAC;QAC1D,sEAAsE;QACtE,OAAO,IAAI,MAAM,CACf,IAAI,GAAG,CACL,MAAM;aACH,QAAQ,CACP,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW,IAAI,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,CAClE;aACA,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,WAAW,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC;YACzC,EAAE;YACF,EAAE,WAAW,EAAE,WAAW,EAAE;SAC7B,CAAC,CACL,CACF,CAAC;IACJ,CAAC;CACF"}
package/dist/index.d.ts CHANGED
@@ -1,3 +1,4 @@
1
+ export { CoalescingDecisionModel } from './coalescing-decision-model.ts';
1
2
  export { DecisionScorer } from './decision-scorer.ts';
2
3
  export { MissingVectorError, UnknownArticleError } from './errors.ts';
3
4
  export { ExpectedFThreshold } from './expected-f-threshold.ts';
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EAAE,kBAAkB,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAC/D,YAAY,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AACpD,YAAY,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAC3D,OAAO,EAAE,KAAK,YAAY,EAAE,KAAK,aAAa,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAC5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,OAAO,EAAE,KAAK,MAAM,EAAE,KAAK,MAAM,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAC9E,YAAY,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAChD,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,YAAY,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,uBAAuB,EAAE,MAAM,gCAAgC,CAAC;AACzE,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EAAE,kBAAkB,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAC/D,YAAY,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AACpD,YAAY,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAC3D,OAAO,EAAE,KAAK,YAAY,EAAE,KAAK,aAAa,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAC5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,OAAO,EAAE,KAAK,MAAM,EAAE,KAAK,MAAM,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAC9E,YAAY,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAChD,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,YAAY,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC"}
package/dist/index.js CHANGED
@@ -1,3 +1,4 @@
1
+ export { CoalescingDecisionModel } from "./coalescing-decision-model.js";
1
2
  export { DecisionScorer } from "./decision-scorer.js";
2
3
  export { MissingVectorError, UnknownArticleError } from "./errors.js";
3
4
  export { ExpectedFThreshold } from "./expected-f-threshold.js";
package/dist/index.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EAAE,kBAAkB,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAE/D,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAEpD,OAAO,EAAyC,MAAM,EAAE,MAAM,aAAa,CAAC;AAC5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,OAAO,EAA4B,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAE9E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,uBAAuB,EAAE,MAAM,gCAAgC,CAAC;AACzE,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EAAE,kBAAkB,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAE/D,OAAO,EAAE,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAEpD,OAAO,EAAyC,MAAM,EAAE,MAAM,aAAa,CAAC;AAC5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,OAAO,EAA4B,aAAa,EAAE,MAAM,qBAAqB,CAAC;AAE9E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC"}
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@zukhruf/relevance-filtering",
3
- "version": "0.6.2",
3
+ "version": "0.6.4",
4
4
  "description": "Ranks a stream of articles for a reader's profile through a scorer port (a decision model, or embedding similarity calibrated on the reader's answers), with thresholds that turn the probabilities into yes or no.",
5
5
  "license": "MIT",
6
6
  "repository": {
@@ -44,6 +44,6 @@
44
44
  },
45
45
  "devDependencies": {
46
46
  "@stdlib/blas-ext-base-gsumpw": "^0.3.1",
47
- "@zukhruf/embeddings": "^0.6.2"
47
+ "@zukhruf/embeddings": "^0.6.4"
48
48
  }
49
49
  }