@zukhruf/relevance-filtering 0.4.2 → 0.5.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (50) hide show
  1. package/README.md +110 -52
  2. package/dist/binormal.d.ts +1 -15
  3. package/dist/binormal.d.ts.map +1 -1
  4. package/dist/binormal.js +6 -12
  5. package/dist/binormal.js.map +1 -1
  6. package/dist/decision-scorer.d.ts +12 -0
  7. package/dist/decision-scorer.d.ts.map +1 -0
  8. package/dist/decision-scorer.js +29 -0
  9. package/dist/decision-scorer.js.map +1 -0
  10. package/dist/expected-f-threshold.d.ts +10 -0
  11. package/dist/expected-f-threshold.d.ts.map +1 -0
  12. package/dist/expected-f-threshold.js +33 -0
  13. package/dist/expected-f-threshold.js.map +1 -0
  14. package/dist/index.d.ts +8 -4
  15. package/dist/index.d.ts.map +1 -1
  16. package/dist/index.js +6 -2
  17. package/dist/index.js.map +1 -1
  18. package/dist/profile.d.ts +1 -16
  19. package/dist/profile.d.ts.map +1 -1
  20. package/dist/profile.js +3 -88
  21. package/dist/profile.js.map +1 -1
  22. package/dist/scorer.d.ts +12 -0
  23. package/dist/scorer.d.ts.map +1 -0
  24. package/dist/scorer.js +2 -0
  25. package/dist/scorer.js.map +1 -0
  26. package/dist/scores.d.ts +23 -0
  27. package/dist/scores.d.ts.map +1 -0
  28. package/dist/scores.js +51 -0
  29. package/dist/scores.js.map +1 -0
  30. package/dist/similarity-scorer.d.ts +16 -0
  31. package/dist/similarity-scorer.d.ts.map +1 -0
  32. package/dist/similarity-scorer.js +109 -0
  33. package/dist/similarity-scorer.js.map +1 -0
  34. package/dist/threshold.d.ts +6 -0
  35. package/dist/threshold.d.ts.map +1 -0
  36. package/dist/threshold.js +2 -0
  37. package/dist/threshold.js.map +1 -0
  38. package/dist/utility-threshold.d.ts +10 -0
  39. package/dist/utility-threshold.d.ts.map +1 -0
  40. package/dist/utility-threshold.js +22 -0
  41. package/dist/utility-threshold.js.map +1 -0
  42. package/package.json +7 -4
  43. package/dist/expected-f.d.ts +0 -2
  44. package/dist/expected-f.d.ts.map +0 -1
  45. package/dist/expected-f.js +0 -23
  46. package/dist/expected-f.js.map +0 -1
  47. package/dist/relevance-filter.d.ts +0 -47
  48. package/dist/relevance-filter.d.ts.map +0 -1
  49. package/dist/relevance-filter.js +0 -100
  50. package/dist/relevance-filter.js.map +0 -1
package/README.md CHANGED
@@ -1,19 +1,61 @@
1
1
  # @zukhruf/relevance-filtering
2
2
 
3
- A reader wants the new articles of a feed that are about an interest, and not the others. This package ranks the articles of a stream for a profile of the reader, the best first, so a reader can show the best N articles of each day. A profile has a name, one sentence that describes the interest, optional keywords, and the yes and no answers of the reader. The package compares each article with the sentence by embedding similarity. It makes each similarity a calibrated probability, adds the evidence of the keywords, and ranks the articles by that probability. This kind of program is also known as routing, as in TREC. The package also has a yes/no decision, which is not recommended (see "Rank, do not decide").
3
+ A reader wants the new articles of a feed that are about an interest, and not the others. This package ranks the articles of a stream for a profile of the reader, the best first, so a reader can show the best N articles of each day. A profile has a name, one sentence that describes the interest, optional keywords, and the yes and no answers of the reader. A scorer gives each article a probability and ranks the stream by it. This kind of program is also known as routing, as in TREC. A threshold that the host chooses can turn the probabilities into a yes or no for an action, for example a notification.
4
+
5
+ The package has two scorers:
6
+
7
+ - **`DecisionScorer`** asks a decision model, through the AI SDK, if the main subject of each article is the description. It ranked best in research rounds 13 and 14.
8
+ - **`SimilarityScorer`** compares each article with the description by embedding similarity, makes each similarity a calibrated probability from the reader's answers, and adds the evidence of the keywords. It needs no decision model.
4
9
 
5
10
  The words in these documents have one meaning each. See the glossary in [CONTEXT.md](./CONTEXT.md).
6
11
 
7
12
  ## Use it
8
13
 
9
- This project is an experiment. The package does not start a model, a server or a database. Give it a vectors port that holds the vectors of the articles, for example a `Collection` of [`@zukhruf/embeddings`](../../embeddings), and the vector of the description from the same model.
10
-
11
- See the [recipes](#recipes) for whole programs. The design comes from research round 6 of #3195, and it was frozen before its exam ([ADR 0002](./docs/adr/0002-the-design-is-frozen-by-research-round-6.md)). It was measured with EmbeddingGemma 2, with the document prompt for the articles and the query prompt for the description.
14
+ This project is an experiment. The package does not start a model, a server or a database. See the [recipes](#recipes) for whole programs.
12
15
 
13
16
  ```sh
14
- npm install @zukhruf/relevance-filtering @zukhruf/embeddings ai
17
+ npm install @zukhruf/relevance-filtering ai
15
18
  ```
16
19
 
20
+ ### With a decision model
21
+
22
+ Give the scorer an AI SDK decision model, for example Liquid AI's d1-2B on a llama.cpp server through `@ai-sdk/typesafe-ai`.
23
+
24
+ ```ts
25
+ import { createTypeSafeAi } from '@ai-sdk/typesafe-ai';
26
+
27
+ import {
28
+ DecisionScorer,
29
+ Profile,
30
+ UtilityThreshold,
31
+ } from '@zukhruf/relevance-filtering';
32
+
33
+ const scorer = new DecisionScorer({
34
+ model: createTypeSafeAi({
35
+ baseURL: 'http://127.0.0.1:9931/v1',
36
+ apiKey: 'none',
37
+ }).decisionModel('LiquidAI/d1-2B-GGUF:Q8_0'),
38
+ });
39
+ const profile = new Profile({
40
+ name: 'Security',
41
+ description: 'Cybersecurity: attacks, breaches and the tools that stop them.',
42
+ relevant: [],
43
+ notRelevant: [],
44
+ });
45
+
46
+ const scores = await scorer.score(profile, newArticles); // [{ id, publishedAt, text: `${title} ${lead}` }, …]
47
+
48
+ // The best 5 articles of one day, the best first.
49
+ scores.ranked({ from: dayStart, to: dayEnd }).slice(0, 5);
50
+ scores.probability(id); // from 0 to 1
51
+ // Notify only when a miss costs three times as much as a false alarm.
52
+ scores.accepted(new UtilityThreshold({ falseAlarm: 1, miss: 3 }));
53
+ ```
54
+
55
+ ### With embeddings
56
+
57
+ Give the scorer a vectors port that holds the vectors of the articles, for example a `Collection` of [`@zukhruf/embeddings`](../../embeddings), and a function that gives the vector of a description from the same model. The design comes from research round 6 of #3195, and it was frozen before its exam ([ADR 0002](./docs/adr/0002-the-design-is-frozen-by-research-round-6.md)). It was measured with EmbeddingGemma 2, with the document prompt for the articles and the query prompt for the description.
58
+
17
59
  ```ts
18
60
  import { DatabaseSync } from 'node:sqlite';
19
61
 
@@ -21,7 +63,7 @@ import { createOpenAICompatible } from '@ai-sdk/openai-compatible';
21
63
  import { embed } from 'ai';
22
64
 
23
65
  import { Collection } from '@zukhruf/embeddings';
24
- import { Profile } from '@zukhruf/relevance-filtering';
66
+ import { Profile, SimilarityScorer } from '@zukhruf/relevance-filtering';
25
67
 
26
68
  const model = createOpenAICompatible({
27
69
  name: 'llama.cpp',
@@ -32,6 +74,17 @@ const collection = new Collection({ database, model });
32
74
  // The host upserts each article with the document prompt of the model:
33
75
  // title: <title> | text: <title> <lead>.
34
76
 
77
+ const scorer = new SimilarityScorer({
78
+ vectors: collection,
79
+ embedDescription: async (description) =>
80
+ (
81
+ await embed({
82
+ model,
83
+ value: `task: search result | query: ${description}`,
84
+ })
85
+ ).embedding,
86
+ taughtText: (id) => textOf(id),
87
+ });
35
88
  const profile = new Profile({
36
89
  name: 'Security',
37
90
  description: 'Cybersecurity: attacks, breaches and the tools that stop them.',
@@ -39,21 +92,9 @@ const profile = new Profile({
39
92
  relevant: ['ransomware-hospital'],
40
93
  notRelevant: ['chip-plant'],
41
94
  });
42
- const { embedding } = await embed({
43
- model,
44
- value: `task: search result | query: ${profile.description}`,
45
- });
46
-
47
- const filter = profile.train({
48
- vectors: collection,
49
- descriptionVector: embedding,
50
- stream: newArticles, // [{ id, publishedAt, text: `${title} ${lead}` }, …]
51
- taughtText: (id) => textOf(id),
52
- });
53
95
 
54
- // The best 5 articles of one day, the best first.
55
- filter.ranked({ from: dayStart, to: dayEnd }).slice(0, 5);
56
- filter.probability(id); // from 0 to 1
96
+ const scores = await scorer.score(profile, newArticles);
97
+ scores.ranked({ from: dayStart, to: dayEnd }).slice(0, 5);
57
98
  ```
58
99
 
59
100
  ## API
@@ -61,70 +102,87 @@ filter.probability(id); // from 0 to 1
61
102
  - `new Profile({ name, description, keywords?, relevant, notRelevant })` makes a profile. `relevant` and `notRelevant` are the ids of the articles that the reader answered yes and no for. An id in both lists is a `ConflictingAnswerError`. A profile does no I/O ([ADR 0001](./docs/adr/0001-a-profile-is-a-value-and-train-decides-on-one-stream.md)).
62
103
  - `profile.with(changes)` gives a copy with the changed fields, for example a new answer. The profile does not change.
63
104
  - `keywords` holds strings and regular expressions. A string matches at the start of a word, in any case: `secur` matches "Security" and "securing", but not "cybersecurity". A regular expression matches as it is ([ADR 0003](./docs/adr/0003-a-keyword-is-a-word-start-or-a-regular-expression.md)).
64
- - `profile.train({ vectors, descriptionVector, stream, taughtText })` learns from the answers and decides on the stream. It gives a `CalibratedFilter` or a `KeywordFilter`.
65
- - `vectors` is the port to the vectors of the articles: `query(vector, { from, to })` and `fetch(ids)`. The `Collection` of `@zukhruf/embeddings` has this shape. The stream and the taught articles must be in it.
66
- - `descriptionVector` is the vector of the description, from the model of the stored vectors. Make it with the prompt that the model uses for a query.
67
- - `stream` is a list of one or more articles: `{ id, publishedAt, text }`. `publishedAt` is the date that the vectors store for the article. The keywords are matched in `text`, for example the title and the lead.
68
- - `taughtText(id)` gives the text of a taught relevant article, which the keywords are matched in. The filter calls it only when the profile has keywords.
69
- - A filter gives `ranked(range?)`: the ids of its stream, the best first. With `{ from, to }`, only the articles published in that range, in the same order. The best N of a day are `ranked({ from, to }).slice(0, n)` ([ADR 0004](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md)).
70
- - A filter gives `probability(id)` for each article of its stream.
71
- - A `CalibratedFilter` (`kind: 'calibrated'`) comes from a profile with at least one relevant answer. It ranks by probability, then by similarity, then by id. It also gives its `calibration` (the binormal fit, or Platt's fit).
72
- - A `KeywordFilter` (`kind: 'keywords'`) comes from a profile with no relevant answer, or when the binormal fit cannot hold and the profile has no not-relevant answer. It ranks the articles that match a keyword first, the newest first, and then the other articles by similarity. Its probability is 1 for a match and 0 for the others.
73
- - `accepts(id)`, `accepted()` and the `threshold` of a `CalibratedFilter` give the yes/no decision of the expected-F0.5 threshold. They are not recommended for decisions (see "Rank, do not decide").
74
-
75
- ## Rank, do not decide
76
-
77
- Do not use `accepts`, `accepted` or `threshold` to decide what a reader sees. Research round 9 tested the yes/no decisions on July to September 2025, a slice that no round had seen, and every design lost to the typed keyword: the threshold that a few taught answers set is weak. The ranking was strong, so the user chose a ranked reader on 2026-10-29. Round 10 measured the ranking and froze it. See [the research record](../../../docs/research/relevance-filtering.md) and [ADR 0004](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md).
78
-
79
- ## How it ranks
105
+ - A `Scorer` has `score(profile, stream)`, which gives a `Promise` of `Scores` ([ADR 0005](./docs/adr/0005-a-scorer-and-a-threshold-are-ports.md)). `stream` is a list of one or more articles: `{ id, publishedAt, text }`. `text` is, for example, the title and the lead.
106
+ - `new DecisionScorer({ model })` takes an AI SDK decision model. For each article, it asks the model "Is the main subject of this article: <description>?" about `text`, one article at a time. It uses no keyword and no answer.
107
+ - `new SimilarityScorer({ vectors, embedDescription, taughtText })` is the frozen design.
108
+ - `vectors` is the port to the vectors of the articles: `query(vector, { from, to })` and `fetch(ids)`. The `Collection` of `@zukhruf/embeddings` has this shape. The stream and the taught articles must be in it. `publishedAt` must be the date that the vectors store for the article.
109
+ - `embedDescription(description)` gives the vector of a description, from the model of the stored vectors. Make it with the prompt that the model uses for a query.
110
+ - `taughtText(id)` gives the text of a taught relevant article, which the keywords are matched in. The scorer calls it only when the profile has keywords.
111
+ - A profile with at least one relevant answer gets calibrated probabilities, ranked by probability, then by similarity, then by id. A profile with no relevant answer, or a profile whose binormal fit cannot hold and that has no not-relevant answer, ranks the articles that match a keyword first, the newest first, and then the other articles by similarity. Its probability is 1 for a match and 0 for the others.
112
+ - `Scores` gives `ranked(range?)`: the ids of its stream, the best first. With `{ from, to }`, only the articles published in that range, in the same order. The best N of a day are `ranked({ from, to }).slice(0, n)` ([ADR 0004](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md)).
113
+ - `Scores` gives `probability(id)` for each article of its stream, and `accepted(threshold)`: the articles that the threshold accepts, in the order of `ranked()`.
114
+ - A `Threshold` has `cut(scores, profile)`, which gives the lowest probability to accept (`Infinity` accepts nothing).
115
+ - `new UtilityThreshold({ falseAlarm, miss })` accepts when p >= falseAlarm / (falseAlarm + miss), the Bayes decision rule for calibrated probabilities. The two costs are of 0 or more, and not both 0.
116
+ - `new ExpectedFThreshold({ beta })` accepts the m most probable articles, where m maximises the expected F-beta of the stream. With beta 0.5 it is the threshold of the frozen design.
117
+ - To add a method, implement `Scorer` or `Threshold`. A scorer gives `new Scores({ profile, ranked })`, with `ranked` the stream in its order, each article with its probability.
118
+
119
+ ## Rank, then decide for each action
120
+
121
+ Show the ranking, and decide yes or no only for an action that needs it. Research round 9 tested the yes/no decisions of the frozen design on July to September 2025, and every design lost to the typed keyword: the threshold that a few taught answers set is weak. The ranking was strong, so the user chose a ranked reader on 2026-10-29 ([ADR 0004](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md)). The probabilities of the decision model were calibrated on January to March 2025 (Brier skill +0.726), but the calibration changed with the answer key (+0.356 with a second key, round 14). Choose a threshold for each action from what a miss and a false alarm cost. See [the research record](../../../docs/research/relevance-filtering.md) and [ADR 0005](./docs/adr/0005-a-scorer-and-a-threshold-are-ports.md).
122
+
123
+ ## How the similarity scorer ranks
80
124
 
81
125
  1. **Similarity.** The similarity of each article is the cosine of its vector and the description vector. The answers do not change the similarity.
82
- 2. **Binormal model.** The filter standardises the similarities of the stream by their mean and standard deviation. It fits a relevant normal distribution and an other normal distribution by semi-supervised EM. The taught similarities stay in the component of their answer. The probability of an article is its posterior of the relevant component.
126
+ 2. **Binormal model.** The scorer standardises the similarities of the stream by their mean and standard deviation. It fits a relevant normal distribution and an other normal distribution by semi-supervised EM. The taught similarities stay in the component of their answer. The probability of an article is its posterior of the relevant component.
83
127
  3. **Platt.** When the stream similarities are all equal, or the relevant mean falls to or below the other mean, the binormal fit cannot hold. Then Platt's method on the taught similarities gives the probabilities.
84
128
  4. **Keyword evidence.** When the profile has keywords, naive Bayes adds the log likelihood ratio of a match, or of no match, to the log odds of each article.
85
- 5. **Order.** The filter ranks the stream by the probability, then by the similarity, then by id.
86
- 6. **Threshold (not recommended).** `accepts` accepts the m most probable articles, where m maximises the expected F0.5 of the stream.
129
+ 5. **Order.** The scorer ranks the stream by the probability, then by the similarity, then by id.
87
130
 
88
131
  ## Errors
89
132
 
90
133
  - `ConflictingAnswerError`: one id is a relevant and a not-relevant answer.
91
- - `NoEvidenceError`: the profile has no relevant answer that the filter can learn from and no keyword.
134
+ - `NoEvidenceError`: the similarity scorer got a profile with no relevant answer that it can learn from and no keyword.
92
135
  - `MissingVectorError`: the vectors have no article of the stream with its publish date.
93
- - `UnknownArticleError`: `probability` or `accepts` got an id that is not in the stream.
94
- - `RangeError`: the stream has one id two times.
136
+ - `UnknownArticleError`: `probability` got an id that is not in the stream.
137
+ - `RangeError`: the stream has one id two times, or the costs of a `UtilityThreshold` give no threshold.
95
138
  - The error of the vectors port, for example the `UnknownDocumentError` of `@zukhruf/embeddings`: a taught article is not in the vectors.
139
+ - The errors of the AI SDK, for example `Experimental_DecisionRefusalError` when the decision model refuses a question. The decision scorer gives no probability in place of a refused or failed call.
96
140
 
97
141
  ## Result
98
142
 
99
- Round 10 of the research ranked July to September 2025 (1,617 articles) for each profile, after product teaching: the reader shows the top 5 or 10 articles by similarity and the top 5 or 10 keyword matches, and the person answers yes or no. The score is MAP over the profiles with at least 15 strict relevant articles in the ranked stream:
143
+ Rounds 13 and 14 ranked January to March 2025 (1,370 articles) for each profile. The score is MAP over the profiles with at least 20 strict relevant articles:
144
+
145
+ | Answer key | Decision scorer (d1-2B) | Similarity scorer | The reader's keyword list |
146
+ | ---------------------------------------- | ----------------------- | ----------------- | ------------------------- |
147
+ | Claude annotators (round 13, 7 profiles) | 0.924 | 0.782 | 0.498 |
148
+ | GPT annotators (round 14, 12 profiles) | 0.808 | 0.634 | 0.402 |
149
+
150
+ The 95% interval of (decision scorer - similarity scorer) is [+0.049, +0.298] and [+0.094, +0.278]. The decision scorer was better on each of the 12 profiles of round 14.
151
+
152
+ Round 10 ranked July to September 2025 (1,617 articles) after product teaching: the reader shows the top 5 or 10 articles by similarity and the top 5 or 10 keyword matches, and the person answers yes or no. The score is MAP over the profiles with at least 15 strict relevant articles in the ranked stream:
100
153
 
101
- | Teaching | This package | The reader's keyword list | Broader keywords | Newest first |
102
- | ------------------------------------------- | ------------ | ------------------------- | ---------------- | ------------ |
103
- | The first 14 days of the slice (9 profiles) | 0.717 | 0.514 | 0.486 | 0.065 |
104
- | The three earlier slices (11 profiles) | 0.766 | 0.476 | 0.462 | 0.055 |
154
+ | Teaching | Similarity scorer | The reader's keyword list | Broader keywords | Newest first |
155
+ | ------------------------------------------- | ----------------- | ------------------------- | ---------------- | ------------ |
156
+ | The first 14 days of the slice (9 profiles) | 0.717 | 0.514 | 0.486 | 0.065 |
157
+ | The three earlier slices (11 profiles) | 0.766 | 0.476 | 0.462 | 0.055 |
105
158
 
106
- A keyword list puts the matches first, the newest first. The 95% interval of (package - keyword list) is [+0.082, +0.296] and [+0.215, +0.374]. `nx run relevance-filtering:eval -- ranking` gives these numbers through the package, to 1e-15 ([eval/README.md](./eval/README.md)).
159
+ A keyword list puts the matches first, the newest first. The 95% interval of (similarity scorer - keyword list) is [+0.082, +0.296] and [+0.215, +0.374]. `nx run relevance-filtering:eval -- ranking` gives these numbers through the package, to 1e-15 ([eval/README.md](./eval/README.md)).
107
160
 
108
161
  ## Limits
109
162
 
110
- - The design is valid only with EmbeddingGemma 2, the vector text of event detection (`title: <title> | text: <title>\n<lead>`) for the articles and the query prompt for the description. Another model gives another similarity scale.
163
+ - The decision scorer was measured only with d1-2B (Q8_0), the question of round 13, and the title and the lead as the text. Another model or another question needs a new research round.
164
+ - The decision scorer calls the model once for each article and each profile. The host decides when to score and if it keeps the scores.
165
+ - The answer keys of rounds 13 and 14 come from language models. No person labelled that slice (limitation 16 of the research record).
166
+ - The similarity design is valid only with EmbeddingGemma 2, the vector text of event detection (`title: <title> | text: <title>\n<lead>`) for the articles and the query prompt for the description. Another model gives another similarity scale.
111
167
  - The binormal model is fitted on the stream. A stream of a few articles gives a weak fit. The research streams held 1,600 to 1,700 articles.
112
- - When the answers are only the most similar articles of a large pool, the taught relevant similarities are far above the relevant articles of the stream. Then the probabilities are too low, and the yes/no threshold accepts too few articles. The order stays good.
168
+ - When the answers are only the most similar articles of a large pool, the taught relevant similarities are far above the relevant articles of the stream. Then the probabilities of the similarity scorer are too low, and a threshold accepts too few articles. The order stays good.
113
169
  - With one relevant answer, the relevant standard deviation sits at its floor, and the keyword evidence sets most of the order.
114
170
  - A keyword string matches at the start of a word only. Type each part that you want: `cyber` and `secur` for "cybersecurity".
115
- - The filter reads the vectors of the whole publish range of the stream. A stream that spans a long time reads many vectors.
171
+ - The similarity scorer reads the vectors of the whole publish range of the stream. A stream that spans a long time reads many vectors.
116
172
 
117
173
  ## Decisions
118
174
 
119
- Before you change a step of the design, read ADR 0002, and run `nx run relevance-filtering:eval` before and after the change.
175
+ Before you change a step of the similarity design, read ADR 0002, and run `nx run relevance-filtering:eval` before and after the change.
120
176
 
121
177
  - [ADR 0001: A profile is a value, and train decides on one stream](./docs/adr/0001-a-profile-is-a-value-and-train-decides-on-one-stream.md)
122
178
  - [ADR 0002: The design is frozen by research round 6](./docs/adr/0002-the-design-is-frozen-by-research-round-6.md)
123
179
  - [ADR 0003: A keyword is a word start or a regular expression](./docs/adr/0003-a-keyword-is-a-word-start-or-a-regular-expression.md)
124
180
  - [ADR 0004: The reader ranks, and does not decide yes or no](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md)
181
+ - [ADR 0005: A scorer and a threshold are ports](./docs/adr/0005-a-scorer-and-a-threshold-are-ports.md)
125
182
  - [Code copied from the event-detection eval](./docs/copied-code.md)
126
183
 
127
184
  ## Recipes
128
185
 
186
+ - [Rank a feed with a decision model](./docs/recipes/rank-a-feed-with-a-decision-model.md)
129
187
  - [Show the best articles of each day with a taught profile](./docs/recipes/filter-a-feed-with-a-taught-profile.md)
130
188
  - [Start a profile with keywords, then teach it](./docs/recipes/start-with-keywords-then-teach.md)
@@ -1,16 +1,2 @@
1
- export type BinormalFit = {
2
- readonly share: number;
3
- readonly relevant: {
4
- readonly mean: number;
5
- readonly sd: number;
6
- };
7
- readonly other: {
8
- readonly mean: number;
9
- readonly sd: number;
10
- };
11
- };
12
- export declare function binormal(stream: readonly number[], relevant: readonly number[], notRelevant: readonly number[]): {
13
- readonly probabilities: number[];
14
- readonly fit: BinormalFit;
15
- } | undefined;
1
+ export declare function binormal(stream: readonly number[], relevant: readonly number[], notRelevant: readonly number[]): number[] | undefined;
16
2
  //# sourceMappingURL=binormal.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"binormal.d.ts","sourceRoot":"","sources":["../src/binormal.ts"],"names":[],"mappings":"AAIA,MAAM,MAAM,WAAW,GAAG;IAExB,QAAQ,CAAC,KAAK,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,QAAQ,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;KAAE,CAAC;IAClE,QAAQ,CAAC,KAAK,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;QAAC,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAA;KAAE,CAAC;CAChE,CAAC;AAgBF,wBAAgB,QAAQ,CACtB,MAAM,EAAE,SAAS,MAAM,EAAE,EACzB,QAAQ,EAAE,SAAS,MAAM,EAAE,EAC3B,WAAW,EAAE,SAAS,MAAM,EAAE,GAC7B;IAAE,QAAQ,CAAC,aAAa,EAAE,MAAM,EAAE,CAAC;IAAC,QAAQ,CAAC,GAAG,EAAE,WAAW,CAAA;CAAE,GAAG,SAAS,CA0D7E"}
1
+ {"version":3,"file":"binormal.d.ts","sourceRoot":"","sources":["../src/binormal.ts"],"names":[],"mappings":"AAiBA,wBAAgB,QAAQ,CACtB,MAAM,EAAE,SAAS,MAAM,EAAE,EACzB,QAAQ,EAAE,SAAS,MAAM,EAAE,EAC3B,WAAW,EAAE,SAAS,MAAM,EAAE,GAC7B,MAAM,EAAE,GAAG,SAAS,CAmDtB"}
package/dist/binormal.js CHANGED
@@ -7,10 +7,11 @@ const LOG_ROOT_TWO_PI = 0.5 * Math.log(2 * Math.PI);
7
7
  function logNormal(x, mu, sd) {
8
8
  return -0.5 * ((x - mu) / sd) ** 2 - Math.log(sd) - LOG_ROOT_TWO_PI;
9
9
  }
10
- // A two-normal mixture of the stream's scores, fitted by semi-supervised EM:
11
- // the taught scores stay in their component (docs/adr/0002). Undefined when
12
- // the fit cannot hold: the stream's scores are all equal, or the relevant
13
- // mean is not above the other mean.
10
+ // The probability of each stream score from a two-normal mixture of the
11
+ // stream's scores, standardised by their mean and standard deviation, and
12
+ // fitted by semi-supervised EM: the taught scores stay in their component
13
+ // (docs/adr/0002). Undefined when the fit cannot hold: the stream's scores
14
+ // are all equal, or the relevant mean is not above the other mean.
14
15
  export function binormal(stream, relevant, notRelevant) {
15
16
  const centre = mean(stream);
16
17
  const spread = populationSd(stream);
@@ -48,14 +49,7 @@ export function binormal(stream, relevant, notRelevant) {
48
49
  if (mu1 <= mu0)
49
50
  return undefined;
50
51
  }
51
- return {
52
- probabilities: posterior(),
53
- fit: {
54
- share,
55
- relevant: { mean: mu1, sd: s1 },
56
- other: { mean: mu0, sd: s0 },
57
- },
58
- };
52
+ return posterior();
59
53
  }
60
54
  // The mean and the floored standard deviation of the stream scores with
61
55
  // their weights and of the taught scores with weight 1, over the joined
@@ -1 +1 @@
1
- {"version":3,"file":"binormal.js","sourceRoot":"","sources":["../src/binormal.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,QAAQ,EAAE,IAAI,EAAE,YAAY,EAAE,GAAG,EAAE,MAAM,iBAAiB,CAAC;AAW/E,MAAM,QAAQ,GAAG,IAAI,CAAC;AACtB,MAAM,UAAU,GAAG,GAAG,CAAC;AACvB,MAAM,SAAS,GAAG,IAAI,CAAC;AACvB,MAAM,WAAW,GAAG,IAAI,CAAC;AACzB,MAAM,eAAe,GAAG,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC;AAEpD,SAAS,SAAS,CAAC,CAAS,EAAE,EAAU,EAAE,EAAU;IAClD,OAAO,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,eAAe,CAAC;AACtE,CAAC;AAED,6EAA6E;AAC7E,4EAA4E;AAC5E,0EAA0E;AAC1E,oCAAoC;AACpC,MAAM,UAAU,QAAQ,CACtB,MAAyB,EACzB,QAA2B,EAC3B,WAA8B;IAE9B,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC;IAC5B,MAAM,MAAM,GAAG,YAAY,CAAC,MAAM,CAAC,CAAC;IACpC,IAAI,MAAM,KAAK,CAAC;QAAE,OAAO,SAAS,CAAC;IACnC,MAAM,QAAQ,GAAG,CAAC,KAAa,EAAE,EAAE,CAAC,CAAC,KAAK,GAAG,MAAM,CAAC,GAAG,MAAM,CAAC;IAC9D,MAAM,CAAC,GAAG,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IAC/B,MAAM,EAAE,GAAG,QAAQ,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IAClC,MAAM,EAAE,GAAG,WAAW,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IACrC,MAAM,EAAE,GAAG,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC;IACxB,MAAM,EAAE,GAAG,GAAG,CAAC;IACf,IAAI,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,WAAW,EAAE,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC;IACpD,IAAI,GAAG,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC;IACnB,IAAI,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,EAAE,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC9C,IAAI,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IAClB,IAAI,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC7C,IAAI,GAAG,IAAI,GAAG;QAAE,OAAO,SAAS,CAAC;IACjC,MAAM,SAAS,GAAG,GAAG,EAAE,CACrB,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CACV,QAAQ,CACN,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC;QACb,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC;QACrB,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC;QACnB,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CACxB,CACF,CAAC;IACJ,IAAI,QAAQ,GAAG,CAAC,QAAQ,CAAC;IACzB,KAAK,IAAI,SAAS,GAAG,CAAC,EAAE,SAAS,GAAG,UAAU,EAAE,SAAS,EAAE,EAAE,CAAC;QAC5D,MAAM,CAAC,GAAG,SAAS,EAAE,CAAC;QACtB,MAAM,aAAa,GACjB,GAAG,CACD,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CACV,SAAS,CACP,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,EACvC,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,GAAG,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CAC5C,CACF,CACF;YACD,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CAAC,CAAC;YACzC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC;QAC5C,IAAI,IAAI,CAAC,GAAG,CAAC,aAAa,GAAG,QAAQ,CAAC,GAAG,SAAS;YAAE,MAAM;QAC1D,QAAQ,GAAG,aAAa,CAAC;QACzB,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC;QAC5C,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC;QAC/B,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,QAAQ,CAClB,CAAC,EACD,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,GAAG,KAAK,CAAC,EAC3B,EAAE,CACH,CAAC;QACF,IAAI,GAAG,IAAI,GAAG;YAAE,OAAO,SAAS,CAAC;IACnC,CAAC;IACD,OAAO;QACL,aAAa,EAAE,SAAS,EAAE;QAC1B,GAAG,EAAE;YACH,KAAK;YACL,QAAQ,EAAE,EAAE,IAAI,EAAE,GAAG,EAAE,EAAE,EAAE,EAAE,EAAE;YAC/B,KAAK,EAAE,EAAE,IAAI,EAAE,GAAG,EAAE,EAAE,EAAE,EAAE,EAAE;SAC7B;KACF,CAAC;AACJ,CAAC;AAED,wEAAwE;AACxE,wEAAwE;AACxE,6BAA6B;AAC7B,SAAS,QAAQ,CACf,CAAoB,EACpB,OAA0B,EAC1B,MAAyB;IAEzB,MAAM,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,GAAG,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;IAC/C,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,GAAG,MAAM,CAAC,CAAC;IAC5B,MAAM,KAAK,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IACrB,MAAM,MAAM,GACV,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,GAAG,KAAK,CAAC;IAClE,MAAM,QAAQ,GACZ,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC;QACvE,KAAK,CAAC;IACR,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC,CAAC;AAC3D,CAAC"}
1
+ {"version":3,"file":"binormal.js","sourceRoot":"","sources":["../src/binormal.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,QAAQ,EAAE,IAAI,EAAE,YAAY,EAAE,GAAG,EAAE,MAAM,iBAAiB,CAAC;AAE/E,MAAM,QAAQ,GAAG,IAAI,CAAC;AACtB,MAAM,UAAU,GAAG,GAAG,CAAC;AACvB,MAAM,SAAS,GAAG,IAAI,CAAC;AACvB,MAAM,WAAW,GAAG,IAAI,CAAC;AACzB,MAAM,eAAe,GAAG,GAAG,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC;AAEpD,SAAS,SAAS,CAAC,CAAS,EAAE,EAAU,EAAE,EAAU;IAClD,OAAO,CAAC,GAAG,GAAG,CAAC,CAAC,CAAC,GAAG,EAAE,CAAC,GAAG,EAAE,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,GAAG,CAAC,EAAE,CAAC,GAAG,eAAe,CAAC;AACtE,CAAC;AAED,wEAAwE;AACxE,0EAA0E;AAC1E,0EAA0E;AAC1E,2EAA2E;AAC3E,mEAAmE;AACnE,MAAM,UAAU,QAAQ,CACtB,MAAyB,EACzB,QAA2B,EAC3B,WAA8B;IAE9B,MAAM,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC,CAAC;IAC5B,MAAM,MAAM,GAAG,YAAY,CAAC,MAAM,CAAC,CAAC;IACpC,IAAI,MAAM,KAAK,CAAC;QAAE,OAAO,SAAS,CAAC;IACnC,MAAM,QAAQ,GAAG,CAAC,KAAa,EAAE,EAAE,CAAC,CAAC,KAAK,GAAG,MAAM,CAAC,GAAG,MAAM,CAAC;IAC9D,MAAM,CAAC,GAAG,MAAM,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IAC/B,MAAM,EAAE,GAAG,QAAQ,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IAClC,MAAM,EAAE,GAAG,WAAW,CAAC,GAAG,CAAC,QAAQ,CAAC,CAAC;IACrC,MAAM,EAAE,GAAG,CAAC,GAAG,CAAC,CAAC,MAAM,CAAC;IACxB,MAAM,EAAE,GAAG,GAAG,CAAC;IACf,IAAI,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,WAAW,EAAE,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC;IACpD,IAAI,GAAG,GAAG,IAAI,CAAC,EAAE,CAAC,CAAC;IACnB,IAAI,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,EAAE,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC9C,IAAI,GAAG,GAAG,IAAI,CAAC,CAAC,CAAC,CAAC;IAClB,IAAI,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,YAAY,CAAC,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC;IAC7C,IAAI,GAAG,IAAI,GAAG;QAAE,OAAO,SAAS,CAAC;IACjC,MAAM,SAAS,GAAG,GAAG,EAAE,CACrB,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CACV,QAAQ,CACN,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC;QACb,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC;QACrB,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC;QACnB,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CACxB,CACF,CAAC;IACJ,IAAI,QAAQ,GAAG,CAAC,QAAQ,CAAC;IACzB,KAAK,IAAI,SAAS,GAAG,CAAC,EAAE,SAAS,GAAG,UAAU,EAAE,SAAS,EAAE,EAAE,CAAC;QAC5D,MAAM,CAAC,GAAG,SAAS,EAAE,CAAC;QACtB,MAAM,aAAa,GACjB,GAAG,CACD,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CACV,SAAS,CACP,IAAI,CAAC,GAAG,CAAC,KAAK,CAAC,GAAG,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,EACvC,IAAI,CAAC,GAAG,CAAC,CAAC,GAAG,KAAK,CAAC,GAAG,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CAC5C,CACF,CACF;YACD,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CAAC,CAAC;YACzC,GAAG,CAAC,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,SAAS,CAAC,CAAC,EAAE,GAAG,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC;QAC5C,IAAI,IAAI,CAAC,GAAG,CAAC,aAAa,GAAG,QAAQ,CAAC,GAAG,SAAS;YAAE,MAAM;QAC1D,QAAQ,GAAG,aAAa,CAAC;QACzB,KAAK,GAAG,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC;QAC5C,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,QAAQ,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC;QAC/B,CAAC,GAAG,EAAE,EAAE,CAAC,GAAG,QAAQ,CAClB,CAAC,EACD,CAAC,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC,GAAG,KAAK,CAAC,EAC3B,EAAE,CACH,CAAC;QACF,IAAI,GAAG,IAAI,GAAG;YAAE,OAAO,SAAS,CAAC;IACnC,CAAC;IACD,OAAO,SAAS,EAAE,CAAC;AACrB,CAAC;AAED,wEAAwE;AACxE,wEAAwE;AACxE,6BAA6B;AAC7B,SAAS,QAAQ,CACf,CAAoB,EACpB,OAA0B,EAC1B,MAAyB;IAEzB,MAAM,CAAC,GAAG,CAAC,GAAG,OAAO,EAAE,GAAG,MAAM,CAAC,GAAG,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;IAC/C,MAAM,CAAC,GAAG,CAAC,GAAG,CAAC,EAAE,GAAG,MAAM,CAAC,CAAC;IAC5B,MAAM,KAAK,GAAG,GAAG,CAAC,CAAC,CAAC,CAAC;IACrB,MAAM,MAAM,GACV,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,GAAG,KAAK,CAAC;IAClE,MAAM,QAAQ,GACZ,GAAG,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC;QACvE,KAAK,CAAC;IACR,OAAO,CAAC,MAAM,EAAE,IAAI,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,QAAQ,CAAC,EAAE,QAAQ,CAAC,CAAC,CAAC;AAC3D,CAAC"}
@@ -0,0 +1,12 @@
1
+ import { type Experimental_DecisionModel } from 'ai';
2
+ import type { Profile } from './profile.ts';
3
+ import type { Scorer, Stream } from './scorer.ts';
4
+ import { Scores } from './scores.ts';
5
+ export declare class DecisionScorer implements Scorer {
6
+ #private;
7
+ constructor(options: {
8
+ readonly model: Experimental_DecisionModel;
9
+ });
10
+ score(profile: Profile, stream: Stream): Promise<Scores>;
11
+ }
12
+ //# sourceMappingURL=decision-scorer.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"decision-scorer.d.ts","sourceRoot":"","sources":["../src/decision-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,KAAK,0BAA0B,EAAuB,MAAM,IAAI,CAAC;AAE1E,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAClD,OAAO,EAAE,MAAM,EAAc,MAAM,aAAa,CAAC;AAOjD,qBAAa,cAAe,YAAW,MAAM;;gBAG/B,OAAO,EAAE;QAAE,QAAQ,CAAC,KAAK,EAAE,0BAA0B,CAAA;KAAE;IAI7D,KAAK,CAAC,OAAO,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,MAAM,CAAC;CAoB/D"}
@@ -0,0 +1,29 @@
1
+ import { experimental_decide } from 'ai';
2
+ import { Scores, compareIds } from "./scores.js";
3
+ // A decision model answers a typed question about a text with a calibrated
4
+ // probability, for example Liquid AI's d1-2B. Research rounds 13 and 14
5
+ // (#3383) found that its probability ranks better than the frozen
6
+ // similarity design, with no taught answer and no keyword (docs/adr/0005).
7
+ // The question is the one that round 13 asked, word for word.
8
+ export class DecisionScorer {
9
+ #model;
10
+ constructor(options) {
11
+ this.#model = options.model;
12
+ }
13
+ async score(profile, stream) {
14
+ const instructions = `Is the main subject of this article: ${profile.description}?`;
15
+ const scored = await Array.fromAsync(stream, async ({ id, publishedAt, text }) => {
16
+ const { answers } = await experimental_decide({
17
+ model: this.#model,
18
+ state: text,
19
+ questions: { relevant: { type: 'boolean', instructions } },
20
+ });
21
+ return { id, publishedAt, probability: answers.relevant.probability };
22
+ });
23
+ return new Scores({
24
+ profile,
25
+ ranked: scored.toSorted((a, b) => b.probability - a.probability || compareIds(a.id, b.id)),
26
+ });
27
+ }
28
+ }
29
+ //# sourceMappingURL=decision-scorer.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"decision-scorer.js","sourceRoot":"","sources":["../src/decision-scorer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAmC,mBAAmB,EAAE,MAAM,IAAI,CAAC;AAI1E,OAAO,EAAE,MAAM,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AAEjD,2EAA2E;AAC3E,wEAAwE;AACxE,kEAAkE;AAClE,2EAA2E;AAC3E,8DAA8D;AAC9D,MAAM,OAAO,cAAc;IAChB,MAAM,CAA6B;IAE5C,YAAY,OAAuD;QACjE,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;IAC9B,CAAC;IAED,KAAK,CAAC,KAAK,CAAC,OAAgB,EAAE,MAAc;QAC1C,MAAM,YAAY,GAAG,wCAAwC,OAAO,CAAC,WAAW,GAAG,CAAC;QACpF,MAAM,MAAM,GAAG,MAAM,KAAK,CAAC,SAAS,CAClC,MAAM,EACN,KAAK,EAAE,EAAE,EAAE,EAAE,WAAW,EAAE,IAAI,EAAE,EAAE,EAAE;YAClC,MAAM,EAAE,OAAO,EAAE,GAAG,MAAM,mBAAmB,CAAC;gBAC5C,KAAK,EAAE,IAAI,CAAC,MAAM;gBAClB,KAAK,EAAE,IAAI;gBACX,SAAS,EAAE,EAAE,QAAQ,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,YAAY,EAAE,EAAE;aAC3D,CAAC,CAAC;YACH,OAAO,EAAE,EAAE,EAAE,WAAW,EAAE,WAAW,EAAE,OAAO,CAAC,QAAQ,CAAC,WAAW,EAAE,CAAC;QACxE,CAAC,CACF,CAAC;QACF,OAAO,IAAI,MAAM,CAAC;YAChB,OAAO;YACP,MAAM,EAAE,MAAM,CAAC,QAAQ,CACrB,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW,IAAI,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,CAClE;SACF,CAAC,CAAC;IACL,CAAC;CACF"}
@@ -0,0 +1,10 @@
1
+ import type { Scores } from './scores.ts';
2
+ import type { Threshold } from './threshold.ts';
3
+ export declare class ExpectedFThreshold implements Threshold {
4
+ #private;
5
+ constructor(options: {
6
+ readonly beta: number;
7
+ });
8
+ cut(scores: Scores): number;
9
+ }
10
+ //# sourceMappingURL=expected-f-threshold.d.ts.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"expected-f-threshold.d.ts","sourceRoot":"","sources":["../src/expected-f-threshold.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAE1C,OAAO,KAAK,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAQhD,qBAAa,kBAAmB,YAAW,SAAS;;gBAGtC,OAAO,EAAE;QAAE,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;KAAE;IAI9C,GAAG,CAAC,MAAM,EAAE,MAAM,GAAG,MAAM;CAmB5B"}
@@ -0,0 +1,33 @@
1
+ import { sum } from "./statistics.js";
2
+ // The plug-in rule that maximises the expected F-beta over the probabilities
3
+ // themselves. Accepting the m most probable articles has the expected F-beta
4
+ // (1 + beta²) * (sum of their probabilities) / (beta² * sum of all
5
+ // probabilities + m). With beta 0.5 it is the threshold of the frozen design
6
+ // (docs/adr/0002). When no article has a probability above 0, it accepts
7
+ // nothing.
8
+ export class ExpectedFThreshold {
9
+ #betaSquared;
10
+ constructor(options) {
11
+ this.#betaSquared = options.beta ** 2;
12
+ }
13
+ cut(scores) {
14
+ const order = scores
15
+ .ranked()
16
+ .map((id) => scores.probability(id))
17
+ .toSorted((a, b) => b - a);
18
+ const total = sum(order);
19
+ let best;
20
+ let cumulative = 0;
21
+ for (const [index, probability] of order.entries()) {
22
+ cumulative += probability;
23
+ const value = ((1 + this.#betaSquared) * cumulative) /
24
+ (this.#betaSquared * total + index + 1);
25
+ if (best === undefined || value > best.value)
26
+ best = { value, threshold: probability };
27
+ }
28
+ if (best === undefined || best.value <= 0)
29
+ return Infinity;
30
+ return best.threshold;
31
+ }
32
+ }
33
+ //# sourceMappingURL=expected-f-threshold.js.map
@@ -0,0 +1 @@
1
+ {"version":3,"file":"expected-f-threshold.js","sourceRoot":"","sources":["../src/expected-f-threshold.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,GAAG,EAAE,MAAM,iBAAiB,CAAC;AAGtC,6EAA6E;AAC7E,6EAA6E;AAC7E,mEAAmE;AACnE,6EAA6E;AAC7E,yEAAyE;AACzE,WAAW;AACX,MAAM,OAAO,kBAAkB;IACpB,YAAY,CAAS;IAE9B,YAAY,OAAkC;QAC5C,IAAI,CAAC,YAAY,GAAG,OAAO,CAAC,IAAI,IAAI,CAAC,CAAC;IACxC,CAAC;IAED,GAAG,CAAC,MAAc;QAChB,MAAM,KAAK,GAAG,MAAM;aACjB,MAAM,EAAE;aACR,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,MAAM,CAAC,WAAW,CAAC,EAAE,CAAC,CAAC;aACnC,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC;QAC7B,MAAM,KAAK,GAAG,GAAG,CAAC,KAAK,CAAC,CAAC;QACzB,IAAI,IAAsD,CAAC;QAC3D,IAAI,UAAU,GAAG,CAAC,CAAC;QACnB,KAAK,MAAM,CAAC,KAAK,EAAE,WAAW,CAAC,IAAI,KAAK,CAAC,OAAO,EAAE,EAAE,CAAC;YACnD,UAAU,IAAI,WAAW,CAAC;YAC1B,MAAM,KAAK,GACT,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,YAAY,CAAC,GAAG,UAAU,CAAC;gBACtC,CAAC,IAAI,CAAC,YAAY,GAAG,KAAK,GAAG,KAAK,GAAG,CAAC,CAAC,CAAC;YAC1C,IAAI,IAAI,KAAK,SAAS,IAAI,KAAK,GAAG,IAAI,CAAC,KAAK;gBAC1C,IAAI,GAAG,EAAE,KAAK,EAAE,SAAS,EAAE,WAAW,EAAE,CAAC;QAC7C,CAAC;QACD,IAAI,IAAI,KAAK,SAAS,IAAI,IAAI,CAAC,KAAK,IAAI,CAAC;YAAE,OAAO,QAAQ,CAAC;QAC3D,OAAO,IAAI,CAAC,SAAS,CAAC;IACxB,CAAC;CACF"}
package/dist/index.d.ts CHANGED
@@ -1,8 +1,12 @@
1
- export type { BinormalFit } from './binormal.ts';
1
+ export { DecisionScorer } from './decision-scorer.ts';
2
2
  export { ConflictingAnswerError, MissingVectorError, NoEvidenceError, UnknownArticleError, } from './errors.ts';
3
+ export { ExpectedFThreshold } from './expected-f-threshold.ts';
3
4
  export type { Keyword } from './keyword.ts';
4
- export type { PlattFit } from './platt.ts';
5
- export { type Article, Profile, type ProfileFields, type TrainOptions, } from './profile.ts';
6
- export { type Calibration, CalibratedFilter, KeywordFilter, type PublishRange, type RankedArticle, type RelevanceFilter, } from './relevance-filter.ts';
5
+ export { Profile, type ProfileFields } from './profile.ts';
6
+ export type { Article, Scorer, Stream } from './scorer.ts';
7
+ export { type PublishRange, type ScoredArticle, Scores } from './scores.ts';
8
+ export { SimilarityScorer } from './similarity-scorer.ts';
9
+ export type { Threshold } from './threshold.ts';
10
+ export { UtilityThreshold } from './utility-threshold.ts';
7
11
  export type { Similarity, Vectors } from './vectors.ts';
8
12
  //# sourceMappingURL=index.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AACjD,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AACrB,YAAY,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAC5C,YAAY,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAC3C,OAAO,EACL,KAAK,OAAO,EACZ,OAAO,EACP,KAAK,aAAa,EAClB,KAAK,YAAY,GAClB,MAAM,cAAc,CAAC;AACtB,OAAO,EACL,KAAK,WAAW,EAChB,gBAAgB,EAChB,aAAa,EACb,KAAK,YAAY,EACjB,KAAK,aAAa,EAClB,KAAK,eAAe,GACrB,MAAM,uBAAuB,CAAC;AAC/B,YAAY,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC"}
1
+ {"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAC/D,YAAY,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,OAAO,EAAE,KAAK,aAAa,EAAE,MAAM,cAAc,CAAC;AAC3D,YAAY,EAAE,OAAO,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAC3D,OAAO,EAAE,KAAK,YAAY,EAAE,KAAK,aAAa,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAC5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,YAAY,EAAE,SAAS,EAAE,MAAM,gBAAgB,CAAC;AAChD,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAC1D,YAAY,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC"}
package/dist/index.js CHANGED
@@ -1,4 +1,8 @@
1
+ export { DecisionScorer } from "./decision-scorer.js";
1
2
  export { ConflictingAnswerError, MissingVectorError, NoEvidenceError, UnknownArticleError, } from "./errors.js";
2
- export { Profile, } from "./profile.js";
3
- export { CalibratedFilter, KeywordFilter, } from "./relevance-filter.js";
3
+ export { ExpectedFThreshold } from "./expected-f-threshold.js";
4
+ export { Profile } from "./profile.js";
5
+ export { Scores } from "./scores.js";
6
+ export { SimilarityScorer } from "./similarity-scorer.js";
7
+ export { UtilityThreshold } from "./utility-threshold.js";
4
8
  //# sourceMappingURL=index.js.map
package/dist/index.js.map CHANGED
@@ -1 +1 @@
1
- {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AAGrB,OAAO,EAEL,OAAO,GAGR,MAAM,cAAc,CAAC;AACtB,OAAO,EAEL,gBAAgB,EAChB,aAAa,GAId,MAAM,uBAAuB,CAAC"}
1
+ {"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,cAAc,EAAE,MAAM,sBAAsB,CAAC;AACtD,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,kBAAkB,EAAE,MAAM,2BAA2B,CAAC;AAE/D,OAAO,EAAE,OAAO,EAAsB,MAAM,cAAc,CAAC;AAE3D,OAAO,EAAyC,MAAM,EAAE,MAAM,aAAa,CAAC;AAC5E,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC;AAE1D,OAAO,EAAE,gBAAgB,EAAE,MAAM,wBAAwB,CAAC"}
package/dist/profile.d.ts CHANGED
@@ -1,6 +1,4 @@
1
- import { type Keyword } from './keyword.ts';
2
- import { type RelevanceFilter } from './relevance-filter.ts';
3
- import type { Vectors } from './vectors.ts';
1
+ import type { Keyword } from './keyword.ts';
4
2
  export type ProfileFields = {
5
3
  readonly name: string;
6
4
  readonly description: string;
@@ -8,19 +6,7 @@ export type ProfileFields = {
8
6
  readonly relevant: readonly string[];
9
7
  readonly notRelevant: readonly string[];
10
8
  };
11
- export type Article = {
12
- readonly id: string;
13
- readonly publishedAt: Date;
14
- readonly text: string;
15
- };
16
- export type TrainOptions = {
17
- readonly vectors: Vectors;
18
- readonly descriptionVector: readonly number[];
19
- readonly stream: readonly [Article, ...Article[]];
20
- readonly taughtText: (id: string) => string;
21
- };
22
9
  export declare class Profile {
23
- #private;
24
10
  readonly name: string;
25
11
  readonly description: string;
26
12
  readonly keywords: readonly Keyword[];
@@ -28,6 +14,5 @@ export declare class Profile {
28
14
  readonly notRelevant: readonly string[];
29
15
  constructor(fields: ProfileFields);
30
16
  with(changes: Partial<ProfileFields>): Profile;
31
- train(options: TrainOptions): RelevanceFilter;
32
17
  }
33
18
  //# sourceMappingURL=profile.d.ts.map
@@ -1 +1 @@
1
- {"version":3,"file":"profile.d.ts","sourceRoot":"","sources":["../src/profile.ts"],"names":[],"mappings":"AAQA,OAAO,EAAE,KAAK,OAAO,EAAc,MAAM,cAAc,CAAC;AAExD,OAAO,EAKL,KAAK,eAAe,EACrB,MAAM,uBAAuB,CAAC;AAC/B,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAE5C,MAAM,MAAM,aAAa,GAAG;IAE1B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAGtB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,QAAQ,CAAC,EAAE,SAAS,OAAO,EAAE,CAAC;IAEvC,QAAQ,CAAC,QAAQ,EAAE,SAAS,MAAM,EAAE,CAAC;IACrC,QAAQ,CAAC,WAAW,EAAE,SAAS,MAAM,EAAE,CAAC;CACzC,CAAC;AAGF,MAAM,MAAM,OAAO,GAAG;IACpB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IAEpB,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;IAG3B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;CACvB,CAAC;AAEF,MAAM,MAAM,YAAY,GAAG;IAEzB,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAG1B,QAAQ,CAAC,iBAAiB,EAAE,SAAS,MAAM,EAAE,CAAC;IAG9C,QAAQ,CAAC,MAAM,EAAE,SAAS,CAAC,OAAO,EAAE,GAAG,OAAO,EAAE,CAAC,CAAC;IAGlD,QAAQ,CAAC,UAAU,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,MAAM,CAAC;CAC7C,CAAC;AAKF,qBAAa,OAAO;;IAClB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC,QAAQ,CAAC,QAAQ,EAAE,SAAS,MAAM,EAAE,CAAC;IACrC,QAAQ,CAAC,WAAW,EAAE,SAAS,MAAM,EAAE,CAAC;gBAE5B,MAAM,EAAE,aAAa;IAYjC,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC,aAAa,CAAC,GAAG,OAAO;IAa9C,KAAK,CAAC,OAAO,EAAE,YAAY,GAAG,eAAe;CAwD9C"}
1
+ {"version":3,"file":"profile.d.ts","sourceRoot":"","sources":["../src/profile.ts"],"names":[],"mappings":"AACA,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAE5C,MAAM,MAAM,aAAa,GAAG;IAE1B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAEtB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,QAAQ,CAAC,EAAE,SAAS,OAAO,EAAE,CAAC;IAEvC,QAAQ,CAAC,QAAQ,EAAE,SAAS,MAAM,EAAE,CAAC;IACrC,QAAQ,CAAC,WAAW,EAAE,SAAS,MAAM,EAAE,CAAC;CACzC,CAAC;AAMF,qBAAa,OAAO;IAClB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC,QAAQ,CAAC,QAAQ,EAAE,SAAS,MAAM,EAAE,CAAC;IACrC,QAAQ,CAAC,WAAW,EAAE,SAAS,MAAM,EAAE,CAAC;gBAE5B,MAAM,EAAE,aAAa;IAYjC,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC,aAAa,CAAC,GAAG,OAAO;CAU/C"}