@zukhruf/relevance-filtering 0.3.54 → 0.3.55
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +25 -17
- package/dist/index.d.ts +1 -1
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js.map +1 -1
- package/dist/profile.d.ts.map +1 -1
- package/dist/profile.js +15 -5
- package/dist/profile.js.map +1 -1
- package/dist/relevance-filter.d.ts +20 -2
- package/dist/relevance-filter.d.ts.map +1 -1
- package/dist/relevance-filter.js +60 -17
- package/dist/relevance-filter.js.map +1 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @zukhruf/relevance-filtering
|
|
2
2
|
|
|
3
|
-
A reader wants the new articles of a feed that are about an interest, and not the others. This package
|
|
3
|
+
A reader wants the new articles of a feed that are about an interest, and not the others. This package ranks the articles of a stream for a profile of the reader, the best first, so a reader can show the best N articles of each day. A profile has a name, one sentence that describes the interest, optional keywords, and the yes and no answers of the reader. The package compares each article with the sentence by embedding similarity. It makes each similarity a calibrated probability, adds the evidence of the keywords, and ranks the articles by that probability. This kind of program is also known as routing, as in TREC. The package also has a yes/no decision, which is not recommended (see "Rank, do not decide").
|
|
4
4
|
|
|
5
5
|
The words in these documents have one meaning each. See the glossary in [CONTEXT.md](./CONTEXT.md).
|
|
6
6
|
|
|
@@ -51,7 +51,8 @@ const filter = profile.train({
|
|
|
51
51
|
taughtText: (id) => textOf(id),
|
|
52
52
|
});
|
|
53
53
|
|
|
54
|
-
|
|
54
|
+
// The best 5 articles of one day, the best first.
|
|
55
|
+
filter.ranked({ from: dayStart, to: dayEnd }).slice(0, 5);
|
|
55
56
|
filter.probability(id); // from 0 to 1
|
|
56
57
|
```
|
|
57
58
|
|
|
@@ -65,17 +66,24 @@ filter.probability(id); // from 0 to 1
|
|
|
65
66
|
- `descriptionVector` is the vector of the description, from the model of the stored vectors. Make it with the prompt that the model uses for a query.
|
|
66
67
|
- `stream` is a list of one or more articles: `{ id, publishedAt, text }`. `publishedAt` is the date that the vectors store for the article. The keywords are matched in `text`, for example the title and the lead.
|
|
67
68
|
- `taughtText(id)` gives the text of a taught relevant article, which the keywords are matched in. The filter calls it only when the profile has keywords.
|
|
68
|
-
- A filter gives `
|
|
69
|
-
- A
|
|
70
|
-
- A `
|
|
69
|
+
- A filter gives `ranked(range?)`: the ids of its stream, the best first. With `{ from, to }`, only the articles published in that range, in the same order. The best N of a day are `ranked({ from, to }).slice(0, n)` ([ADR 0004](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md)).
|
|
70
|
+
- A filter gives `probability(id)` for each article of its stream.
|
|
71
|
+
- A `CalibratedFilter` (`kind: 'calibrated'`) comes from a profile with at least one relevant answer. It ranks by probability, then by similarity, then by id. It also gives its `calibration` (the binormal fit, or Platt's fit).
|
|
72
|
+
- A `KeywordFilter` (`kind: 'keywords'`) comes from a profile with no relevant answer, or when the binormal fit cannot hold and the profile has no not-relevant answer. It ranks the articles that match a keyword first, the newest first, and then the other articles by similarity. Its probability is 1 for a match and 0 for the others.
|
|
73
|
+
- `accepts(id)`, `accepted()` and the `threshold` of a `CalibratedFilter` give the yes/no decision of the expected-F0.5 threshold. They are not recommended for decisions (see "Rank, do not decide").
|
|
71
74
|
|
|
72
|
-
##
|
|
75
|
+
## Rank, do not decide
|
|
76
|
+
|
|
77
|
+
Do not use `accepts`, `accepted` or `threshold` to decide what a reader sees. Research round 9 tested the yes/no decisions on July to September 2025, a slice that no round had seen, and every design lost to the typed keyword: the threshold that a few taught answers set is weak. The ranking was strong, so the user chose a ranked reader on 2026-10-29. Round 10 measured the ranking and froze it. See [the research record](../../../docs/research/relevance-filtering.md) and [ADR 0004](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md).
|
|
78
|
+
|
|
79
|
+
## How it ranks
|
|
73
80
|
|
|
74
81
|
1. **Similarity.** The similarity of each article is the cosine of its vector and the description vector. The answers do not change the similarity.
|
|
75
82
|
2. **Binormal model.** The filter standardises the similarities of the stream by their mean and standard deviation. It fits a relevant normal distribution and an other normal distribution by semi-supervised EM. The taught similarities stay in the component of their answer. The probability of an article is its posterior of the relevant component.
|
|
76
83
|
3. **Platt.** When the stream similarities are all equal, or the relevant mean falls to or below the other mean, the binormal fit cannot hold. Then Platt's method on the taught similarities gives the probabilities.
|
|
77
84
|
4. **Keyword evidence.** When the profile has keywords, naive Bayes adds the log likelihood ratio of a match, or of no match, to the log odds of each article.
|
|
78
|
-
5. **
|
|
85
|
+
5. **Order.** The filter ranks the stream by the probability, then by the similarity, then by id.
|
|
86
|
+
6. **Threshold (not recommended).** `accepts` accepts the m most probable articles, where m maximises the expected F0.5 of the stream.
|
|
79
87
|
|
|
80
88
|
## Errors
|
|
81
89
|
|
|
@@ -88,22 +96,21 @@ filter.probability(id); // from 0 to 1
|
|
|
88
96
|
|
|
89
97
|
## Result
|
|
90
98
|
|
|
91
|
-
Round
|
|
99
|
+
Round 10 of the research ranked July to September 2025 (1,617 articles) for each profile, after product teaching: the reader shows the top 5 or 10 articles by similarity and the top 5 or 10 keyword matches, and the person answers yes or no. The score is MAP over the profiles with at least 15 strict relevant articles in the ranked stream:
|
|
92
100
|
|
|
93
|
-
|
|
|
94
|
-
|
|
|
95
|
-
|
|
|
96
|
-
| The
|
|
97
|
-
| Broader keywords | 0.528 | none |
|
|
98
|
-
| Accept every article | 0.067 | none |
|
|
101
|
+
| Teaching | This package | The reader's keyword list | Broader keywords | Newest first |
|
|
102
|
+
| ------------------------------------------- | ------------ | ------------------------- | ---------------- | ------------ |
|
|
103
|
+
| The first 14 days of the slice (9 profiles) | 0.717 | 0.514 | 0.486 | 0.065 |
|
|
104
|
+
| The three earlier slices (11 profiles) | 0.766 | 0.476 | 0.462 | 0.055 |
|
|
99
105
|
|
|
100
|
-
The 95% interval of (package - keyword) is [
|
|
106
|
+
A keyword list puts the matches first, the newest first. The 95% interval of (package - keyword list) is [+0.082, +0.296] and [+0.215, +0.374]. `nx run relevance-filtering:eval -- ranking` gives these numbers through the package, to 1e-15 ([eval/README.md](./eval/README.md)).
|
|
101
107
|
|
|
102
108
|
## Limits
|
|
103
109
|
|
|
104
110
|
- The design is valid only with EmbeddingGemma 2, the vector text of event grouping (`title: <title> | text: <title>\n<lead>`) for the articles and the query prompt for the description. Another model gives another similarity scale.
|
|
105
111
|
- The binormal model is fitted on the stream. A stream of a few articles gives a weak fit. The research streams held 1,600 to 1,700 articles.
|
|
106
|
-
- When the answers are only the most similar articles of a large pool, the taught relevant similarities are far above the relevant articles of the stream. Then the
|
|
112
|
+
- When the answers are only the most similar articles of a large pool, the taught relevant similarities are far above the relevant articles of the stream. Then the probabilities are too low, and the yes/no threshold accepts too few articles. The order stays good.
|
|
113
|
+
- With one relevant answer, the relevant standard deviation sits at its floor, and the keyword evidence sets most of the order.
|
|
107
114
|
- A keyword string matches at the start of a word only. Type each part that you want: `cyber` and `secur` for "cybersecurity".
|
|
108
115
|
- The filter reads the vectors of the whole publish range of the stream. A stream that spans a long time reads many vectors.
|
|
109
116
|
|
|
@@ -114,9 +121,10 @@ Before you change a step of the design, read ADR 0002, and run `nx run relevance
|
|
|
114
121
|
- [ADR 0001: A profile is a value, and train decides on one stream](./docs/adr/0001-a-profile-is-a-value-and-train-decides-on-one-stream.md)
|
|
115
122
|
- [ADR 0002: The design is frozen by research round 6](./docs/adr/0002-the-design-is-frozen-by-research-round-6.md)
|
|
116
123
|
- [ADR 0003: A keyword is a word start or a regular expression](./docs/adr/0003-a-keyword-is-a-word-start-or-a-regular-expression.md)
|
|
124
|
+
- [ADR 0004: The reader ranks, and does not decide yes or no](./docs/adr/0004-the-reader-ranks-and-does-not-decide-yes-or-no.md)
|
|
117
125
|
- [Code copied from the event-grouping eval](./docs/copied-code.md)
|
|
118
126
|
|
|
119
127
|
## Recipes
|
|
120
128
|
|
|
121
|
-
- [
|
|
129
|
+
- [Show the best articles of each day with a taught profile](./docs/recipes/filter-a-feed-with-a-taught-profile.md)
|
|
122
130
|
- [Start a profile with keywords, then teach it](./docs/recipes/start-with-keywords-then-teach.md)
|
package/dist/index.d.ts
CHANGED
|
@@ -3,6 +3,6 @@ export { ConflictingAnswerError, MissingVectorError, NoEvidenceError, UnknownArt
|
|
|
3
3
|
export type { Keyword } from './keyword.ts';
|
|
4
4
|
export type { PlattFit } from './platt.ts';
|
|
5
5
|
export { type Article, Profile, type ProfileFields, type TrainOptions, } from './profile.ts';
|
|
6
|
-
export { type Calibration, CalibratedFilter, KeywordFilter, type RelevanceFilter, } from './relevance-filter.ts';
|
|
6
|
+
export { type Calibration, CalibratedFilter, KeywordFilter, type PublishRange, type RankedArticle, type RelevanceFilter, } from './relevance-filter.ts';
|
|
7
7
|
export type { Similarity, Vectors } from './vectors.ts';
|
|
8
8
|
//# sourceMappingURL=index.d.ts.map
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AACjD,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AACrB,YAAY,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAC5C,YAAY,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAC3C,OAAO,EACL,KAAK,OAAO,EACZ,OAAO,EACP,KAAK,aAAa,EAClB,KAAK,YAAY,GAClB,MAAM,cAAc,CAAC;AACtB,OAAO,EACL,KAAK,WAAW,EAChB,gBAAgB,EAChB,aAAa,EACb,KAAK,eAAe,GACrB,MAAM,uBAAuB,CAAC;AAC/B,YAAY,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC"}
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA,YAAY,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AACjD,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AACrB,YAAY,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAC5C,YAAY,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAC3C,OAAO,EACL,KAAK,OAAO,EACZ,OAAO,EACP,KAAK,aAAa,EAClB,KAAK,YAAY,GAClB,MAAM,cAAc,CAAC;AACtB,OAAO,EACL,KAAK,WAAW,EAChB,gBAAgB,EAChB,aAAa,EACb,KAAK,YAAY,EACjB,KAAK,aAAa,EAClB,KAAK,eAAe,GACrB,MAAM,uBAAuB,CAAC;AAC/B,YAAY,EAAE,UAAU,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC"}
|
package/dist/index.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AAGrB,OAAO,EAEL,OAAO,GAGR,MAAM,cAAc,CAAC;AACtB,OAAO,EAEL,gBAAgB,EAChB,aAAa,
|
|
1
|
+
{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AACA,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,EACf,mBAAmB,GACpB,MAAM,aAAa,CAAC;AAGrB,OAAO,EAEL,OAAO,GAGR,MAAM,cAAc,CAAC;AACtB,OAAO,EAEL,gBAAgB,EAChB,aAAa,GAId,MAAM,uBAAuB,CAAC"}
|
package/dist/profile.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"profile.d.ts","sourceRoot":"","sources":["../src/profile.ts"],"names":[],"mappings":"AAQA,OAAO,EAAE,KAAK,OAAO,EAAc,MAAM,cAAc,CAAC;AAExD,OAAO,
|
|
1
|
+
{"version":3,"file":"profile.d.ts","sourceRoot":"","sources":["../src/profile.ts"],"names":[],"mappings":"AAQA,OAAO,EAAE,KAAK,OAAO,EAAc,MAAM,cAAc,CAAC;AAExD,OAAO,EAKL,KAAK,eAAe,EACrB,MAAM,uBAAuB,CAAC;AAC/B,OAAO,KAAK,EAAE,OAAO,EAAE,MAAM,cAAc,CAAC;AAE5C,MAAM,MAAM,aAAa,GAAG;IAE1B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IAGtB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,QAAQ,CAAC,EAAE,SAAS,OAAO,EAAE,CAAC;IAEvC,QAAQ,CAAC,QAAQ,EAAE,SAAS,MAAM,EAAE,CAAC;IACrC,QAAQ,CAAC,WAAW,EAAE,SAAS,MAAM,EAAE,CAAC;CACzC,CAAC;AAGF,MAAM,MAAM,OAAO,GAAG;IACpB,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IAEpB,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;IAG3B,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;CACvB,CAAC;AAEF,MAAM,MAAM,YAAY,GAAG;IAEzB,QAAQ,CAAC,OAAO,EAAE,OAAO,CAAC;IAG1B,QAAQ,CAAC,iBAAiB,EAAE,SAAS,MAAM,EAAE,CAAC;IAG9C,QAAQ,CAAC,MAAM,EAAE,SAAS,CAAC,OAAO,EAAE,GAAG,OAAO,EAAE,CAAC,CAAC;IAGlD,QAAQ,CAAC,UAAU,EAAE,CAAC,EAAE,EAAE,MAAM,KAAK,MAAM,CAAC;CAC7C,CAAC;AAKF,qBAAa,OAAO;;IAClB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,WAAW,EAAE,MAAM,CAAC;IAC7B,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAC;IACtC,QAAQ,CAAC,QAAQ,EAAE,SAAS,MAAM,EAAE,CAAC;IACrC,QAAQ,CAAC,WAAW,EAAE,SAAS,MAAM,EAAE,CAAC;gBAE5B,MAAM,EAAE,aAAa;IAYjC,IAAI,CAAC,OAAO,EAAE,OAAO,CAAC,aAAa,CAAC,GAAG,OAAO;IAa9C,KAAK,CAAC,OAAO,EAAE,YAAY,GAAG,eAAe;CAwD9C"}
|
package/dist/profile.js
CHANGED
|
@@ -36,15 +36,21 @@ export class Profile {
|
|
|
36
36
|
...changes,
|
|
37
37
|
});
|
|
38
38
|
}
|
|
39
|
-
// Learns from the answers and decides on the stream
|
|
39
|
+
// Learns from the answers and ranks and decides on the stream
|
|
40
|
+
// (docs/adr/0002, docs/adr/0004).
|
|
40
41
|
train(options) {
|
|
41
42
|
const { stream } = options;
|
|
42
43
|
if (new Set(stream.map(({ id }) => id)).size !== stream.length)
|
|
43
44
|
throw new RangeError('Each article of the stream must have its own id.');
|
|
45
|
+
const scores = streamScores(options);
|
|
46
|
+
const ranked = stream.map(({ id, publishedAt }, index) => ({
|
|
47
|
+
id,
|
|
48
|
+
publishedAt,
|
|
49
|
+
similarity: scores[index] ?? 0,
|
|
50
|
+
}));
|
|
44
51
|
const matches = () => stream.map(({ text }) => matchesAny(this.keywords, text));
|
|
45
52
|
if (this.relevant.length === 0)
|
|
46
|
-
return this.#keywordFilter(
|
|
47
|
-
const scores = streamScores(options);
|
|
53
|
+
return this.#keywordFilter(ranked, matches);
|
|
48
54
|
const taught = taughtScores(options, this.relevant, this.notRelevant);
|
|
49
55
|
const fitted = binormal(scores, taught.relevant, taught.notRelevant);
|
|
50
56
|
let probabilities;
|
|
@@ -59,10 +65,11 @@ export class Profile {
|
|
|
59
65
|
calibration = fit;
|
|
60
66
|
}
|
|
61
67
|
else
|
|
62
|
-
return this.#keywordFilter(
|
|
68
|
+
return this.#keywordFilter(ranked, matches);
|
|
63
69
|
if (this.keywords.length > 0)
|
|
64
70
|
probabilities = withKeywordEvidence(probabilities, matches(), this.relevant.map((id) => matchesAny(this.keywords, options.taughtText(id))));
|
|
65
71
|
return new CalibratedFilter({
|
|
72
|
+
stream: ranked,
|
|
66
73
|
probabilities: new Map(stream.map(({ id }, index) => [id, probabilities[index] ?? 0])),
|
|
67
74
|
calibration,
|
|
68
75
|
threshold: expectedF05Threshold(probabilities),
|
|
@@ -72,7 +79,10 @@ export class Profile {
|
|
|
72
79
|
if (this.keywords.length === 0)
|
|
73
80
|
throw new NoEvidenceError(this.name);
|
|
74
81
|
const matched = matches();
|
|
75
|
-
return new KeywordFilter(
|
|
82
|
+
return new KeywordFilter({
|
|
83
|
+
stream,
|
|
84
|
+
matches: new Map(stream.map(({ id }, index) => [id, matched[index] ?? false])),
|
|
85
|
+
});
|
|
76
86
|
}
|
|
77
87
|
}
|
|
78
88
|
// The similarity of the description to each article of the stream, in the
|
package/dist/profile.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"profile.js","sourceRoot":"","sources":["../src/profile.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AACzC,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,GAChB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,mBAAmB,EAAE,MAAM,eAAe,CAAC;AACpD,OAAO,EAAE,oBAAoB,EAAE,MAAM,iBAAiB,CAAC;AACvD,OAAO,EAAgB,UAAU,EAAE,MAAM,cAAc,CAAC;AACxD,OAAO,EAAE,KAAK,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AACrD,OAAO,EACL,gBAAgB,EAEhB,aAAa,
|
|
1
|
+
{"version":3,"file":"profile.js","sourceRoot":"","sources":["../src/profile.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,QAAQ,EAAE,MAAM,eAAe,CAAC;AACzC,OAAO,EACL,sBAAsB,EACtB,kBAAkB,EAClB,eAAe,GAChB,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,mBAAmB,EAAE,MAAM,eAAe,CAAC;AACpD,OAAO,EAAE,oBAAoB,EAAE,MAAM,iBAAiB,CAAC;AACvD,OAAO,EAAgB,UAAU,EAAE,MAAM,cAAc,CAAC;AACxD,OAAO,EAAE,KAAK,EAAE,gBAAgB,EAAE,MAAM,YAAY,CAAC;AACrD,OAAO,EACL,gBAAgB,EAEhB,aAAa,GAGd,MAAM,uBAAuB,CAAC;AAuC/B,wEAAwE;AACxE,0EAA0E;AAC1E,uCAAuC;AACvC,MAAM,OAAO,OAAO;IACT,IAAI,CAAS;IACb,WAAW,CAAS;IACpB,QAAQ,CAAqB;IAC7B,QAAQ,CAAoB;IAC5B,WAAW,CAAoB;IAExC,YAAY,MAAqB;QAC/B,MAAM,QAAQ,GAAG,IAAI,GAAG,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC;QAC1C,MAAM,QAAQ,GAAG,MAAM,CAAC,WAAW,CAAC,IAAI,CAAC,CAAC,EAAE,EAAE,EAAE,CAAC,QAAQ,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC;QACnE,IAAI,QAAQ,KAAK,SAAS;YAAE,MAAM,IAAI,sBAAsB,CAAC,QAAQ,CAAC,CAAC;QACvE,IAAI,CAAC,IAAI,GAAG,MAAM,CAAC,IAAI,CAAC;QACxB,IAAI,CAAC,WAAW,GAAG,MAAM,CAAC,WAAW,CAAC;QACtC,IAAI,CAAC,QAAQ,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,QAAQ,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC;QAC5D,IAAI,CAAC,QAAQ,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC;QACpD,IAAI,CAAC,WAAW,GAAG,MAAM,CAAC,MAAM,CAAC,CAAC,GAAG,MAAM,CAAC,WAAW,CAAC,CAAC,CAAC;IAC5D,CAAC;IAED,uDAAuD;IACvD,IAAI,CAAC,OAA+B;QAClC,OAAO,IAAI,OAAO,CAAC;YACjB,IAAI,EAAE,IAAI,CAAC,IAAI;YACf,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,QAAQ,EAAE,IAAI,CAAC,QAAQ;YACvB,WAAW,EAAE,IAAI,CAAC,WAAW;YAC7B,GAAG,OAAO;SACX,CAAC,CAAC;IACL,CAAC;IAED,8DAA8D;IAC9D,kCAAkC;IAClC,KAAK,CAAC,OAAqB;QACzB,MAAM,EAAE,MAAM,EAAE,GAAG,OAAO,CAAC;QAC3B,IAAI,IAAI,GAAG,CAAC,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,CAAC,CAAC,IAAI,KAAK,MAAM,CAAC,MAAM;YAC5D,MAAM,IAAI,UAAU,CAAC,kDAAkD,CAAC,CAAC;QAC3E,MAAM,MAAM,GAAG,YAAY,CAAC,OAAO,CAAC,CAAC;QACrC,MAAM,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,WAAW,EAAE,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC;YACzD,EAAE;YACF,WAAW;YACX,UAAU,EAAE,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC;SAC/B,CAAC,CAAC,CAAC;QACJ,MAAM,OAAO,GAAG,GAAG,EAAE,CACnB,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,IAAI,EAAE,EAAE,EAAE,CAAC,UAAU,CAAC,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,CAAC,CAAC;QAC5D,IAAI,IAAI,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC;YAAE,OAAO,IAAI,CAAC,cAAc,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;QAC5E,MAAM,MAAM,GAAG,YAAY,CAAC,OAAO,EAAE,IAAI,CAAC,QAAQ,EAAE,IAAI,CAAC,WAAW,CAAC,CAAC;QACtE,MAAM,MAAM,GAAG,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,WAAW,CAAC,CAAC;QACrE,IAAI,aAAuB,CAAC;QAC5B,IAAI,WAAwB,CAAC;QAC7B,IAAI,MAAM,KAAK,SAAS,EAAE,CAAC;YACzB,aAAa,GAAG,MAAM,CAAC,aAAa,CAAC;YACrC,WAAW,GAAG,EAAE,MAAM,EAAE,UAAU,EAAE,GAAG,MAAM,CAAC,GAAG,EAAE,CAAC;QACtD,CAAC;aAAM,IAAI,IAAI,CAAC,WAAW,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACvC,MAAM,GAAG,GAAG,KAAK,CAAC,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,WAAW,CAAC,CAAC;YACvD,aAAa,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,KAAK,EAAE,EAAE,CAAC,gBAAgB,CAAC,GAAG,EAAE,KAAK,CAAC,CAAC,CAAC;YACpE,WAAW,GAAG,GAAG,CAAC;QACpB,CAAC;;YAAM,OAAO,IAAI,CAAC,cAAc,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;QACnD,IAAI,IAAI,CAAC,QAAQ,CAAC,MAAM,GAAG,CAAC;YAC1B,aAAa,GAAG,mBAAmB,CACjC,aAAa,EACb,OAAO,EAAE,EACT,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,CACvB,UAAU,CAAC,IAAI,CAAC,QAAQ,EAAE,OAAO,CAAC,UAAU,CAAC,EAAE,CAAC,CAAC,CAClD,CACF,CAAC;QACJ,OAAO,IAAI,gBAAgB,CAAC;YAC1B,MAAM,EAAE,MAAM;YACd,aAAa,EAAE,IAAI,GAAG,CACpB,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC,EAAE,EAAE,aAAa,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,CAC/D;YACD,WAAW;YACX,SAAS,EAAE,oBAAoB,CAAC,aAAa,CAAC;SAC/C,CAAC,CAAC;IACL,CAAC;IAED,cAAc,CACZ,MAAgC,EAChC,OAAwB;QAExB,IAAI,IAAI,CAAC,QAAQ,CAAC,MAAM,KAAK,CAAC;YAAE,MAAM,IAAI,eAAe,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;QACrE,MAAM,OAAO,GAAG,OAAO,EAAE,CAAC;QAC1B,OAAO,IAAI,aAAa,CAAC;YACvB,MAAM;YACN,OAAO,EAAE,IAAI,GAAG,CACd,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,EAAE,KAAK,EAAE,EAAE,CAAC,CAAC,EAAE,EAAE,OAAO,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,CAAC,CAC7D;SACF,CAAC,CAAC;IACL,CAAC;CACF;AAED,0EAA0E;AAC1E,4EAA4E;AAC5E,SAAS,YAAY,CAAC,EAAE,OAAO,EAAE,iBAAiB,EAAE,MAAM,EAAgB;IACxE,MAAM,KAAK,GAAG,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,WAAW,EAAE,EAAE,EAAE,CAAC,WAAW,CAAC,OAAO,EAAE,CAAC,CAAC;IACrE,MAAM,YAAY,GAAG,IAAI,GAAG,CAC1B,OAAO;SACJ,KAAK,CAAC,iBAAiB,EAAE;QACxB,IAAI,EAAE,IAAI,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;QACtD,EAAE,EAAE,IAAI,IAAI,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC;KACrD,CAAC;SACD,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,UAAU,EAAE,EAAE,EAAE,CAAC,CAAC,EAAE,EAAE,UAAU,CAAC,CAAC,CACjD,CAAC;IACF,OAAO,MAAM,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE;QAC3B,MAAM,UAAU,GAAG,YAAY,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QACxC,IAAI,UAAU,KAAK,SAAS;YAAE,MAAM,IAAI,kBAAkB,CAAC,EAAE,CAAC,CAAC;QAC/D,OAAO,UAAU,CAAC;IACpB,CAAC,CAAC,CAAC;AACL,CAAC;AAED,0EAA0E;AAC1E,oEAAoE;AACpE,SAAS,YAAY,CACnB,EAAE,OAAO,EAAE,iBAAiB,EAAgB,EAC5C,QAA2B,EAC3B,WAA8B;IAE9B,MAAM,MAAM,GAAG,IAAI,CAAC,KAAK,CAAC,GAAG,iBAAiB,CAAC,CAAC;IAChD,MAAM,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC,CAAC,GAAG,QAAQ,EAAE,GAAG,WAAW,CAAC,CAAC,CAAC;IAC5D,MAAM,KAAK,GAAG,CAAC,EAAU,EAAE,EAAE;QAC3B,MAAM,MAAM,GAAG,MAAM,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,EAAE,CAAC;QACpC,IAAI,KAAK,GAAG,CAAC,CAAC;QACd,KAAK,IAAI,KAAK,GAAG,CAAC,EAAE,KAAK,GAAG,iBAAiB,CAAC,MAAM,EAAE,KAAK,EAAE;YAC3D,KAAK;gBACH,CAAC,MAAM,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,iBAAiB,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,CAAC;QACtE,OAAO,KAAK,CAAC;IACf,CAAC,CAAC;IACF,OAAO,EAAE,QAAQ,EAAE,QAAQ,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,WAAW,EAAE,WAAW,CAAC,GAAG,CAAC,KAAK,CAAC,EAAE,CAAC;AAChF,CAAC"}
|
|
@@ -3,10 +3,24 @@ import type { PlattFit } from './platt.ts';
|
|
|
3
3
|
export type Calibration = ({
|
|
4
4
|
readonly method: 'binormal';
|
|
5
5
|
} & BinormalFit) | PlattFit;
|
|
6
|
+
export type RankedArticle = {
|
|
7
|
+
readonly id: string;
|
|
8
|
+
readonly publishedAt: Date;
|
|
9
|
+
readonly similarity: number;
|
|
10
|
+
};
|
|
11
|
+
export type PublishRange = {
|
|
12
|
+
readonly from: Date;
|
|
13
|
+
readonly to: Date;
|
|
14
|
+
};
|
|
6
15
|
declare abstract class Decisions {
|
|
7
16
|
#private;
|
|
8
|
-
constructor(
|
|
17
|
+
constructor(options: {
|
|
18
|
+
readonly probabilities: ReadonlyMap<string, number>;
|
|
19
|
+
readonly accepted: ReadonlySet<string>;
|
|
20
|
+
readonly order: readonly RankedArticle[];
|
|
21
|
+
});
|
|
9
22
|
probability(id: string): number;
|
|
23
|
+
ranked(range?: PublishRange): string[];
|
|
10
24
|
accepts(id: string): boolean;
|
|
11
25
|
accepted(): string[];
|
|
12
26
|
}
|
|
@@ -15,6 +29,7 @@ export declare class CalibratedFilter extends Decisions {
|
|
|
15
29
|
readonly calibration: Calibration;
|
|
16
30
|
readonly threshold: number | undefined;
|
|
17
31
|
constructor(options: {
|
|
32
|
+
readonly stream: readonly RankedArticle[];
|
|
18
33
|
readonly probabilities: ReadonlyMap<string, number>;
|
|
19
34
|
readonly calibration: Calibration;
|
|
20
35
|
readonly threshold: number | undefined;
|
|
@@ -22,7 +37,10 @@ export declare class CalibratedFilter extends Decisions {
|
|
|
22
37
|
}
|
|
23
38
|
export declare class KeywordFilter extends Decisions {
|
|
24
39
|
readonly kind = "keywords";
|
|
25
|
-
constructor(
|
|
40
|
+
constructor(options: {
|
|
41
|
+
readonly stream: readonly RankedArticle[];
|
|
42
|
+
readonly matches: ReadonlyMap<string, boolean>;
|
|
43
|
+
});
|
|
26
44
|
}
|
|
27
45
|
export type RelevanceFilter = CalibratedFilter | KeywordFilter;
|
|
28
46
|
export {};
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"relevance-filter.d.ts","sourceRoot":"","sources":["../src/relevance-filter.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAEjD,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAG3C,MAAM,MAAM,WAAW,GACnB,CAAC;IAAE,QAAQ,CAAC,MAAM,EAAE,UAAU,CAAA;CAAE,GAAG,WAAW,CAAC,GAG/C,QAAQ,CAAC;AAGb,uBAAe,SAAS;;
|
|
1
|
+
{"version":3,"file":"relevance-filter.d.ts","sourceRoot":"","sources":["../src/relevance-filter.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,eAAe,CAAC;AAEjD,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAG3C,MAAM,MAAM,WAAW,GACnB,CAAC;IAAE,QAAQ,CAAC,MAAM,EAAE,UAAU,CAAA;CAAE,GAAG,WAAW,CAAC,GAG/C,QAAQ,CAAC;AAGb,MAAM,MAAM,aAAa,GAAG;IAC1B,QAAQ,CAAC,EAAE,EAAE,MAAM,CAAC;IACpB,QAAQ,CAAC,WAAW,EAAE,IAAI,CAAC;IAE3B,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAC;CAC7B,CAAC;AAGF,MAAM,MAAM,YAAY,GAAG;IAAE,QAAQ,CAAC,IAAI,EAAE,IAAI,CAAC;IAAC,QAAQ,CAAC,EAAE,EAAE,IAAI,CAAA;CAAE,CAAC;AAGtE,uBAAe,SAAS;;gBAKV,OAAO,EAAE;QACnB,QAAQ,CAAC,aAAa,EAAE,WAAW,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QACpD,QAAQ,CAAC,QAAQ,EAAE,WAAW,CAAC,MAAM,CAAC,CAAC;QACvC,QAAQ,CAAC,KAAK,EAAE,SAAS,aAAa,EAAE,CAAC;KAC1C;IAOD,WAAW,CAAC,EAAE,EAAE,MAAM,GAAG,MAAM;IAS/B,MAAM,CAAC,KAAK,CAAC,EAAE,YAAY,GAAG,MAAM,EAAE;IAatC,OAAO,CAAC,EAAE,EAAE,MAAM,GAAG,OAAO;IAQ5B,QAAQ,IAAI,MAAM,EAAE;CAKrB;AAID,qBAAa,gBAAiB,SAAQ,SAAS;IAC7C,QAAQ,CAAC,IAAI,gBAAgB;IAC7B,QAAQ,CAAC,WAAW,EAAE,WAAW,CAAC;IAGlC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,SAAS,CAAC;gBAE3B,OAAO,EAAE;QACnB,QAAQ,CAAC,MAAM,EAAE,SAAS,aAAa,EAAE,CAAC;QAC1C,QAAQ,CAAC,aAAa,EAAE,WAAW,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;QACpD,QAAQ,CAAC,WAAW,EAAE,WAAW,CAAC;QAClC,QAAQ,CAAC,SAAS,EAAE,MAAM,GAAG,SAAS,CAAC;KACxC;CAsBF;AAMD,qBAAa,aAAc,SAAQ,SAAS;IAC1C,QAAQ,CAAC,IAAI,cAAc;gBAEf,OAAO,EAAE;QACnB,QAAQ,CAAC,MAAM,EAAE,SAAS,aAAa,EAAE,CAAC;QAC1C,QAAQ,CAAC,OAAO,EAAE,WAAW,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;KAChD;CAmBF;AAED,MAAM,MAAM,eAAe,GAAG,gBAAgB,GAAG,aAAa,CAAC"}
|
package/dist/relevance-filter.js
CHANGED
|
@@ -3,9 +3,11 @@ import { UnknownArticleError } from "./errors.js";
|
|
|
3
3
|
class Decisions {
|
|
4
4
|
#probabilities;
|
|
5
5
|
#accepted;
|
|
6
|
-
|
|
7
|
-
|
|
8
|
-
this.#
|
|
6
|
+
#order;
|
|
7
|
+
constructor(options) {
|
|
8
|
+
this.#probabilities = options.probabilities;
|
|
9
|
+
this.#accepted = options.accepted;
|
|
10
|
+
this.#order = options.order;
|
|
9
11
|
}
|
|
10
12
|
// The probability that the article is relevant to the profile.
|
|
11
13
|
probability(id) {
|
|
@@ -14,44 +16,85 @@ class Decisions {
|
|
|
14
16
|
throw new UnknownArticleError(id);
|
|
15
17
|
return probability;
|
|
16
18
|
}
|
|
17
|
-
//
|
|
19
|
+
// The ids of the stream, the best first (docs/adr/0004). With a range, only
|
|
20
|
+
// the articles published in it, in the same order: the best N of a day are
|
|
21
|
+
// ranked({ from, to }).slice(0, n).
|
|
22
|
+
ranked(range) {
|
|
23
|
+
const inRange = range === undefined
|
|
24
|
+
? this.#order
|
|
25
|
+
: this.#order.filter(({ publishedAt }) => publishedAt >= range.from && publishedAt <= range.to);
|
|
26
|
+
return inRange.map(({ id }) => id);
|
|
27
|
+
}
|
|
28
|
+
// True when the expected-F0.5 threshold accepts the article. Not
|
|
29
|
+
// recommended for decisions: use ranked (docs/adr/0004).
|
|
18
30
|
accepts(id) {
|
|
19
31
|
if (!this.#probabilities.has(id))
|
|
20
32
|
throw new UnknownArticleError(id);
|
|
21
33
|
return this.#accepted.has(id);
|
|
22
34
|
}
|
|
23
35
|
// The accepted articles, the most probable first. Equal probabilities keep
|
|
24
|
-
// the order of the stream.
|
|
36
|
+
// the order of the stream. Not recommended for decisions: use ranked
|
|
37
|
+
// (docs/adr/0004).
|
|
25
38
|
accepted() {
|
|
26
39
|
return [...this.#accepted].toSorted((a, b) => this.probability(b) - this.probability(a));
|
|
27
40
|
}
|
|
28
41
|
}
|
|
29
42
|
// A filter that learned from at least one relevant answer. Its probabilities
|
|
30
|
-
// are calibrated
|
|
31
|
-
// F0.5.
|
|
43
|
+
// are calibrated. It ranks by probability, then by similarity, then by id.
|
|
32
44
|
export class CalibratedFilter extends Decisions {
|
|
33
45
|
kind = 'calibrated';
|
|
34
46
|
calibration;
|
|
35
|
-
// The lowest
|
|
47
|
+
// The lowest probability that the expected-F0.5 threshold accepts.
|
|
48
|
+
// Undefined when nothing is accepted.
|
|
36
49
|
threshold;
|
|
37
50
|
constructor(options) {
|
|
38
|
-
const { probabilities, threshold } = options;
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
51
|
+
const { stream, probabilities, threshold } = options;
|
|
52
|
+
const probability = (id) => probabilities.get(id) ?? 0;
|
|
53
|
+
super({
|
|
54
|
+
probabilities,
|
|
55
|
+
accepted: new Set(threshold === undefined
|
|
56
|
+
? []
|
|
57
|
+
: [...probabilities]
|
|
58
|
+
.filter(([, value]) => value >= threshold)
|
|
59
|
+
.map(([id]) => id)),
|
|
60
|
+
order: stream.toSorted((a, b) => probability(b.id) - probability(a.id) ||
|
|
61
|
+
b.similarity - a.similarity ||
|
|
62
|
+
compareIds(a.id, b.id)),
|
|
63
|
+
});
|
|
44
64
|
this.calibration = options.calibration;
|
|
45
65
|
this.threshold = threshold;
|
|
46
66
|
}
|
|
47
67
|
}
|
|
48
68
|
// A filter with no relevant answer to learn from. It accepts the articles
|
|
49
69
|
// that match a keyword of the profile: the probability is 1 for a match and
|
|
50
|
-
// 0 for the others, and it is not calibrated.
|
|
70
|
+
// 0 for the others, and it is not calibrated. It ranks the matches first,
|
|
71
|
+
// the newest first, and then the other articles by similarity.
|
|
51
72
|
export class KeywordFilter extends Decisions {
|
|
52
73
|
kind = 'keywords';
|
|
53
|
-
constructor(
|
|
54
|
-
|
|
74
|
+
constructor(options) {
|
|
75
|
+
const { stream, matches } = options;
|
|
76
|
+
const match = (id) => matches.get(id) ?? false;
|
|
77
|
+
super({
|
|
78
|
+
probabilities: new Map([...matches].map(([id, value]) => [id, value ? 1 : 0])),
|
|
79
|
+
accepted: new Set([...matches].filter(([, value]) => value).map(([id]) => id)),
|
|
80
|
+
order: stream.toSorted((a, b) => {
|
|
81
|
+
if (match(a.id) !== match(b.id))
|
|
82
|
+
return match(a.id) ? -1 : 1;
|
|
83
|
+
const key = match(a.id)
|
|
84
|
+
? b.publishedAt.getTime() - a.publishedAt.getTime()
|
|
85
|
+
: b.similarity - a.similarity;
|
|
86
|
+
return key || compareIds(a.id, b.id);
|
|
87
|
+
}),
|
|
88
|
+
});
|
|
55
89
|
}
|
|
56
90
|
}
|
|
91
|
+
// Compares by code unit, not by locale, so the order is the same on every
|
|
92
|
+
// machine.
|
|
93
|
+
function compareIds(left, right) {
|
|
94
|
+
if (left < right)
|
|
95
|
+
return -1;
|
|
96
|
+
if (left > right)
|
|
97
|
+
return 1;
|
|
98
|
+
return 0;
|
|
99
|
+
}
|
|
57
100
|
//# sourceMappingURL=relevance-filter.js.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"relevance-filter.js","sourceRoot":"","sources":["../src/relevance-filter.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;
|
|
1
|
+
{"version":3,"file":"relevance-filter.js","sourceRoot":"","sources":["../src/relevance-filter.ts"],"names":[],"mappings":"AACA,OAAO,EAAE,mBAAmB,EAAE,MAAM,aAAa,CAAC;AAqBlD,4CAA4C;AAC5C,MAAe,SAAS;IACb,cAAc,CAA8B;IAC5C,SAAS,CAAsB;IAC/B,MAAM,CAA2B;IAE1C,YAAY,OAIX;QACC,IAAI,CAAC,cAAc,GAAG,OAAO,CAAC,aAAa,CAAC;QAC5C,IAAI,CAAC,SAAS,GAAG,OAAO,CAAC,QAAQ,CAAC;QAClC,IAAI,CAAC,MAAM,GAAG,OAAO,CAAC,KAAK,CAAC;IAC9B,CAAC;IAED,+DAA+D;IAC/D,WAAW,CAAC,EAAU;QACpB,MAAM,WAAW,GAAG,IAAI,CAAC,cAAc,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;QAChD,IAAI,WAAW,KAAK,SAAS;YAAE,MAAM,IAAI,mBAAmB,CAAC,EAAE,CAAC,CAAC;QACjE,OAAO,WAAW,CAAC;IACrB,CAAC;IAED,4EAA4E;IAC5E,2EAA2E;IAC3E,oCAAoC;IACpC,MAAM,CAAC,KAAoB;QACzB,MAAM,OAAO,GACX,KAAK,KAAK,SAAS;YACjB,CAAC,CAAC,IAAI,CAAC,MAAM;YACb,CAAC,CAAC,IAAI,CAAC,MAAM,CAAC,MAAM,CAChB,CAAC,EAAE,WAAW,EAAE,EAAE,EAAE,CAClB,WAAW,IAAI,KAAK,CAAC,IAAI,IAAI,WAAW,IAAI,KAAK,CAAC,EAAE,CACvD,CAAC;QACR,OAAO,OAAO,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,EAAE,EAAE,EAAE,CAAC,EAAE,CAAC,CAAC;IACrC,CAAC;IAED,iEAAiE;IACjE,yDAAyD;IACzD,OAAO,CAAC,EAAU;QAChB,IAAI,CAAC,IAAI,CAAC,cAAc,CAAC,GAAG,CAAC,EAAE,CAAC;YAAE,MAAM,IAAI,mBAAmB,CAAC,EAAE,CAAC,CAAC;QACpE,OAAO,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;IAChC,CAAC;IAED,2EAA2E;IAC3E,qEAAqE;IACrE,mBAAmB;IACnB,QAAQ;QACN,OAAO,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,QAAQ,CACjC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,GAAG,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,CACpD,CAAC;IACJ,CAAC;CACF;AAED,6EAA6E;AAC7E,2EAA2E;AAC3E,MAAM,OAAO,gBAAiB,SAAQ,SAAS;IACpC,IAAI,GAAG,YAAY,CAAC;IACpB,WAAW,CAAc;IAClC,mEAAmE;IACnE,sCAAsC;IAC7B,SAAS,CAAqB;IAEvC,YAAY,OAKX;QACC,MAAM,EAAE,MAAM,EAAE,aAAa,EAAE,SAAS,EAAE,GAAG,OAAO,CAAC;QACrD,MAAM,WAAW,GAAG,CAAC,EAAU,EAAE,EAAE,CAAC,aAAa,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,CAAC,CAAC;QAC/D,KAAK,CAAC;YACJ,aAAa;YACb,QAAQ,EAAE,IAAI,GAAG,CACf,SAAS,KAAK,SAAS;gBACrB,CAAC,CAAC,EAAE;gBACJ,CAAC,CAAC,CAAC,GAAG,aAAa,CAAC;qBACf,MAAM,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,EAAE,EAAE,CAAC,KAAK,IAAI,SAAS,CAAC;qBACzC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,EAAE,CAAC,CACzB;YACD,KAAK,EAAE,MAAM,CAAC,QAAQ,CACpB,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CACP,WAAW,CAAC,CAAC,CAAC,EAAE,CAAC,GAAG,WAAW,CAAC,CAAC,CAAC,EAAE,CAAC;gBACrC,CAAC,CAAC,UAAU,GAAG,CAAC,CAAC,UAAU;gBAC3B,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,CACzB;SACF,CAAC,CAAC;QACH,IAAI,CAAC,WAAW,GAAG,OAAO,CAAC,WAAW,CAAC;QACvC,IAAI,CAAC,SAAS,GAAG,SAAS,CAAC;IAC7B,CAAC;CACF;AAED,0EAA0E;AAC1E,4EAA4E;AAC5E,0EAA0E;AAC1E,+DAA+D;AAC/D,MAAM,OAAO,aAAc,SAAQ,SAAS;IACjC,IAAI,GAAG,UAAU,CAAC;IAE3B,YAAY,OAGX;QACC,MAAM,EAAE,MAAM,EAAE,OAAO,EAAE,GAAG,OAAO,CAAC;QACpC,MAAM,KAAK,GAAG,CAAC,EAAU,EAAE,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,IAAI,KAAK,CAAC;QACvD,KAAK,CAAC;YACJ,aAAa,EAAE,IAAI,GAAG,CACpB,CAAC,GAAG,OAAO,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,KAAK,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CACvD;YACD,QAAQ,EAAE,IAAI,GAAG,CACf,CAAC,GAAG,OAAO,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,EAAE,CAAC,CAC5D;YACD,KAAK,EAAE,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE;gBAC9B,IAAI,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,KAAK,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC;oBAAE,OAAO,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;gBAC7D,MAAM,GAAG,GAAG,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC;oBACrB,CAAC,CAAC,CAAC,CAAC,WAAW,CAAC,OAAO,EAAE,GAAG,CAAC,CAAC,WAAW,CAAC,OAAO,EAAE;oBACnD,CAAC,CAAC,CAAC,CAAC,UAAU,GAAG,CAAC,CAAC,UAAU,CAAC;gBAChC,OAAO,GAAG,IAAI,UAAU,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,EAAE,CAAC,CAAC;YACvC,CAAC,CAAC;SACH,CAAC,CAAC;IACL,CAAC;CACF;AAID,0EAA0E;AAC1E,WAAW;AACX,SAAS,UAAU,CAAC,IAAY,EAAE,KAAa;IAC7C,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC,CAAC;IAC5B,IAAI,IAAI,GAAG,KAAK;QAAE,OAAO,CAAC,CAAC;IAC3B,OAAO,CAAC,CAAC;AACX,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zukhruf/relevance-filtering",
|
|
3
|
-
"version": "0.3.
|
|
3
|
+
"version": "0.3.55",
|
|
4
4
|
"description": "Decides which articles of a stream match a reader's profile: embedding similarity to the profile's description, calibrated by a semi-supervised binormal model on the reader's yes and no answers, naive-Bayes keyword evidence and an expected-F0.5 threshold.",
|
|
5
5
|
"license": "MIT",
|
|
6
6
|
"repository": {
|
|
@@ -40,7 +40,7 @@
|
|
|
40
40
|
"access": "public"
|
|
41
41
|
},
|
|
42
42
|
"devDependencies": {
|
|
43
|
-
"@zukhruf/embeddings": "^0.3.
|
|
43
|
+
"@zukhruf/embeddings": "^0.3.55",
|
|
44
44
|
"ai": "^7.0.154"
|
|
45
45
|
}
|
|
46
46
|
}
|