broader-vids 0.1.0 → 0.1.4
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -21
- package/README.md +133 -114
- package/bin/broader-vids.cjs +482 -482
- package/bin/plugin-install.cjs +99 -99
- package/bin/profile-data.cjs +327 -327
- package/bin/runtime-lock.cjs +151 -151
- package/docs/ARCHITECTURE.md +398 -396
- package/docs/REFERENCE.md +117 -26
- package/docs/SCORING.md +501 -501
- package/docs/WORKFLOW.md +447 -443
- package/package.json +51 -48
- package/plugin.json +28 -28
- package/python/asr-requirements.txt +315 -315
- package/python/broader_vids-0.1.4-py3-none-any.whl +0 -0
- package/python/core-requirements.txt +4 -4
- package/python/evidence-requirements.txt +640 -640
- package/skills/broader-vids/SKILL.md +269 -269
- package/skills/broader-vids/agents/openai.yaml +4 -4
- package/skills/broader-vids-clean/SKILL.md +42 -42
- package/skills/broader-vids-clean/agents/openai.yaml +4 -4
- package/skills/broader-vids-dashboard/SKILL.md +69 -69
- package/skills/broader-vids-dashboard/agents/openai.yaml +4 -4
- package/skills/broader-vids-reset/SKILL.md +42 -42
- package/skills/broader-vids-reset/agents/openai.yaml +4 -4
- package/skills/broader-vids-setup/SKILL.md +193 -193
- package/skills/broader-vids-setup/agents/openai.yaml +4 -4
- package/python/broader_vids-0.1.0-py3-none-any.whl +0 -0
package/docs/SCORING.md
CHANGED
|
@@ -1,501 +1,501 @@
|
|
|
1
|
-
# Scoring, calibration, and mathematics
|
|
2
|
-
|
|
3
|
-
Broader Vids learns conditional preferences from explicit feedback.
|
|
4
|
-
It does not estimate an objective quality score for every video.
|
|
5
|
-
Its conclusions apply to the retrieved videos and feature contexts that receive feedback.
|
|
6
|
-
|
|
7
|
-
## Terms
|
|
8
|
-
|
|
9
|
-
| Term | Meaning |
|
|
10
|
-
| --- | --- |
|
|
11
|
-
| Prior | A model assumption before the current observations |
|
|
12
|
-
| Posterior | The distribution after the model includes those observations |
|
|
13
|
-
| Predictor | A measured or derived input to a model |
|
|
14
|
-
| Likelihood | The probability model for observed feedback |
|
|
15
|
-
| Holdout | Feedback reserved for testing before training |
|
|
16
|
-
| Calibration | Comparison between predicted responses and observed responses |
|
|
17
|
-
| Provenance | Information about a video's source and the uploader's contribution |
|
|
18
|
-
| Ordinal response | An ordered category such as Low, Medium, or High |
|
|
19
|
-
|
|
20
|
-
## 1. Eligibility
|
|
21
|
-
|
|
22
|
-
Eligibility checks operate separately from learned preferences.
|
|
23
|
-
The engine requires a verified playable duration greater than sixty seconds and at most ninety minutes.
|
|
24
|
-
The saved evidence quality must reach 60 points out of 100.
|
|
25
|
-
For new assessments without a usable transcript, subtract ten points from quality after adding the duration prior, with a floor of zero.
|
|
26
|
-
The same deduction applies to the learned quality component during ranking.
|
|
27
|
-
Audio quality is not a criterion.
|
|
28
|
-
Captions and frames support the four existing content criteria.
|
|
29
|
-
Only rubric-5 evidence is eligible for reuse or new recommendation work.
|
|
30
|
-
Unsupported evidence remains unchanged. It requires a new evidence run.
|
|
31
|
-
By default, the exact Bilibili tag `视频翻译` excludes a video from final selection.
|
|
32
|
-
Watched and dismissed videos cannot enter a new final list.
|
|
33
|
-
|
|
34
|
-
A learned preference cannot override these checks.
|
|
35
|
-
An unavailable video remains an access failure.
|
|
36
|
-
It does not receive an invented low-quality label.
|
|
37
|
-
|
|
38
|
-
## 2. Community response and screening
|
|
39
|
-
|
|
40
|
-
YouTube uses likes divided by views.
|
|
41
|
-
Bilibili uses likes, coins, and favorites divided by views.
|
|
42
|
-
These measures belong to community response, separate from content quality.
|
|
43
|
-
Before admission to the 100-video buffer, both platforms require views >= 100, likes >= 3, and likes/views >= 0.01.
|
|
44
|
-
Bilibili additionally requires coins/views >= 0.001 and favorites/views >= 0.002.
|
|
45
|
-
Missing required metrics fail admission.
|
|
46
|
-
These hard floors use raw rates before the smoothing described below.
|
|
47
|
-
|
|
48
|
-
For metric count \(c\), views \(v\), and platform baseline \(b\), the adjusted rate is
|
|
49
|
-
|
|
50
|
-
\[
|
|
51
|
-
\widehat r=\frac{c+1000b}{v+1000}.
|
|
52
|
-
\]
|
|
53
|
-
|
|
54
|
-
The baseline is the middle sorted rate when at least ten valid platform observations exist.
|
|
55
|
-
Otherwise, initial baselines are 0.03 for likes, 0.01 for coins, and 0.02 for favorites.
|
|
56
|
-
The implementation limits the baseline below by \(10^{-6}\).
|
|
57
|
-
|
|
58
|
-
The metric score is
|
|
59
|
-
|
|
60
|
-
\[
|
|
61
|
-
e=100\sigma\left(\operatorname{clip}\left(\log\frac{\max(\widehat r,10^{-12})}{b},-8,8\right)\right),
|
|
62
|
-
\qquad \sigma(z)=\frac{1}{1+e^{-z}}.
|
|
63
|
-
\]
|
|
64
|
-
|
|
65
|
-
The rate-score helper assigns 50 to a missing count or zero denominator.
|
|
66
|
-
Such entries fail the hard admission floors and cannot enter a new recommendation shortlist.
|
|
67
|
-
Bilibili averages its three metric scores with equal weights.
|
|
68
|
-
These correlated metrics do not count as independent evidence of quality.
|
|
69
|
-
|
|
70
|
-
Initial screening uses 70% community response and 30% title-anchor support.
|
|
71
|
-
An anchor is a query term found in the returned metadata.
|
|
72
|
-
Collection scans batches of up to 100 unique candidates.
|
|
73
|
-
The eligible pool targets 71 familiar and 29 exploratory retrievals.
|
|
74
|
-
Each batch fills only the remaining slots in its own group, using the highest screening scores.
|
|
75
|
-
Later batches target deficient groups and preserve source attribution through search continuations.
|
|
76
|
-
Duplicates and rejected candidates do not fill slots.
|
|
77
|
-
This balance applies to collection, before inspection selection.
|
|
78
|
-
Screening policy 5 combines 80% of the metadata screening score with 20% learned interest.
|
|
79
|
-
The metadata score retains its 70% community-response and 30% title-anchor weights.
|
|
80
|
-
The learned term uses topic-specific interest when verified title or tag context names that topic.
|
|
81
|
-
Otherwise, it uses a broad-subject estimate when available.
|
|
82
|
-
Unknown context receives a neutral value of 50.
|
|
83
|
-
Repeated mentions of one topic count once, including mentions shared by the title and tags.
|
|
84
|
-
Each topic also counts once when averaging verified depth estimates.
|
|
85
|
-
The screening model uses verified video context and does not infer quality.
|
|
86
|
-
It uses audience depth only when evidence or the user verifies the video level.
|
|
87
|
-
It does not use speculative quality estimates.
|
|
88
|
-
Screening policy 5 retains the 71 familiar and 29 exploratory eligible-pool targets.
|
|
89
|
-
It also retains thirty-five inspection slots and balances query coverage.
|
|
90
|
-
Inspection selection balances query coverage across thirty-five slots.
|
|
91
|
-
Each slot favors the candidate whose retrieval queries have the fewest existing slots.
|
|
92
|
-
For several query origins, the largest query count applies.
|
|
93
|
-
The applicable screening score and canonical ID resolve ties.
|
|
94
|
-
A missing query uses its retrieval family when available. Unknown sources share no invented group.
|
|
95
|
-
This rule permits repeats when fewer queries can supply candidates.
|
|
96
|
-
Pending entries compete on the same score without guaranteed slots.
|
|
97
|
-
A creator exclusion removes eligibility without creating negative feedback.
|
|
98
|
-
Creator IDs determine these exclusions.
|
|
99
|
-
Account names serve only as labels.
|
|
100
|
-
Unresolved creators remain eligible.
|
|
101
|
-
Each round retains a fixed creator-rule snapshot for screening, assessment scheduling, and final selection.
|
|
102
|
-
Rule changes apply to later rounds rather than changing an active or resumed round.
|
|
103
|
-
Manual and account-specific imported rules remain separate from learning parameters.
|
|
104
|
-
Policy 5 saves one learning-model reference when the round first executes.
|
|
105
|
-
Screening, resumed work, previews, and final selection use that reference.
|
|
106
|
-
Feedback saved during a round changes the model used by a later round.
|
|
107
|
-
Unsupported round policies cannot resume. Current rounds retain their saved settings.
|
|
108
|
-
|
|
109
|
-
## 3. Initial quality and interest
|
|
110
|
-
|
|
111
|
-
The assessment uses four content criteria:
|
|
112
|
-
|
|
113
|
-
| Criterion | Initial content weight | Meaning |
|
|
114
|
-
| --- | ---: | --- |
|
|
115
|
-
| Substance | 40% | Useful explanations or substantive content |
|
|
116
|
-
| Directness | 25% | Relevant content without avoidable repetition or filler |
|
|
117
|
-
| Original contribution | 25% | Evidence of the uploader's own contribution |
|
|
118
|
-
| Purposeful visuals | 10% | Visual material that supports the content |
|
|
119
|
-
|
|
120
|
-
Rubric 5 judges substance and sampled focus against the video's purpose.
|
|
121
|
-
Travel films can provide substance through concrete places, activities, and distinctive visual detail.
|
|
122
|
-
Crafts and games can provide substance through visible execution, development, or outcomes.
|
|
123
|
-
Instructional videos still need evidence of their explanations.
|
|
124
|
-
A software demonstration does not establish that it teaches the underlying algorithm.
|
|
125
|
-
Substance includes explanations needed to understand an instructional video.
|
|
126
|
-
For other video types, judge substantial content against the video's purpose.
|
|
127
|
-
A visual demonstration or performance does not need to teach a lesson.
|
|
128
|
-
Silent frames cannot establish musical performance quality.
|
|
129
|
-
Temporal gaps reduce confidence and widen the interval.
|
|
130
|
-
They do not automatically prevent assessment of sampled relevance and focus.
|
|
131
|
-
A title card alone or unrelated scenes can still leave the overall score unknown.
|
|
132
|
-
Unknown sound, factual accuracy, and full-video pacing remain explicit limitations.
|
|
133
|
-
This outcome is an assessment limitation, not a download failure or a low quality rating.
|
|
134
|
-
A course label alone does not reduce quality.
|
|
135
|
-
A short, complete explanation can have high quality.
|
|
136
|
-
Visual assessment includes text readability, useful contrast, framing, and legible diagrams.
|
|
137
|
-
Simple slides and phone recordings can satisfy these checks.
|
|
138
|
-
Caption evidence cannot establish visual quality or AI generation.
|
|
139
|
-
Creator reputation does not guarantee a high grade.
|
|
140
|
-
Existing scorecards remain stored. New work and cache reuse require rubric 5.
|
|
141
|
-
|
|
142
|
-
An assessed criterion has an integer grade from zero to four.
|
|
143
|
-
Its normalized value is \(u_j=\mathrm{grade}_j/4\).
|
|
144
|
-
An unknown criterion uses a prior mean of 0.5 and a separate missingness predictor.
|
|
145
|
-
Its stored grade remains unknown.
|
|
146
|
-
It does not inherit a high grade from another criterion.
|
|
147
|
-
|
|
148
|
-
For duration \(t\) seconds, the derived initial quality index is
|
|
149
|
-
|
|
150
|
-
\[
|
|
151
|
-
d=\min(t/3600,1),\qquad Q_0=95\sum_j w_j u_j+5d.
|
|
152
|
-
\]
|
|
153
|
-
|
|
154
|
-
Length contributes at most five points.
|
|
155
|
-
The contribution rises to one hour and then remains constant.
|
|
156
|
-
These initial weights express a design choice, not a fitted optimum.
|
|
157
|
-
The saved holistic evidence score remains a separate historical observation.
|
|
158
|
-
|
|
159
|
-
The initial interest probability \(I_0\) is 0.75 for a domain related to the user's stated interests.
|
|
160
|
-
It is 0.35 for a known unrelated domain and 0.50 when domains are unknown.
|
|
161
|
-
Search-domain labels describe retrieval context.
|
|
162
|
-
They do not prove the video's semantic topic.
|
|
163
|
-
|
|
164
|
-
New AI topic labels must be short subject names, with at most 120 characters and 12 words.
|
|
165
|
-
The validator rejects obvious citations, instructions, control characters, and editing notes in labels.
|
|
166
|
-
Invalid new output uses the existing bounded assessment fix workflow.
|
|
167
|
-
Read operations omit unusable labels from older scorecards and annotations.
|
|
168
|
-
They preserve raw evidence, numeric quality scores, saved ratings, and explicit user targets.
|
|
169
|
-
This check detects common formatting errors. It does not prove that a topic correctly describes the video.
|
|
170
|
-
|
|
171
|
-
Source links, repost declarations, `中配`, and YouDub disclosures inform originality assessment.
|
|
172
|
-
A source link alone does not prove copying.
|
|
173
|
-
An unfamiliar copyright enum remains uninterpreted.
|
|
174
|
-
The model must distinguish source teaching quality from the uploader's contribution.
|
|
175
|
-
|
|
176
|
-
## 4. Logistic components
|
|
177
|
-
|
|
178
|
-
Current ratings use categorical topic, subject, depth, and quality responses.
|
|
179
|
-
The quality model factors its categorical likelihood into logistic events.
|
|
180
|
-
Each event uses a predictor vector and a proper Gaussian prior:
|
|
181
|
-
|
|
182
|
-
\[
|
|
183
|
-
y_i\sim\operatorname{Bernoulli}(\sigma(o_i+x_i^T\beta)),
|
|
184
|
-
\qquad \beta_j\sim N(0,s_j^2).
|
|
185
|
-
\]
|
|
186
|
-
|
|
187
|
-
Numerical clipping keeps offsets finite.
|
|
188
|
-
It does not limit the final learned probability.
|
|
189
|
-
An offset also enters as a predictor, so learning can cancel that offset.
|
|
190
|
-
Prior standard deviations are 0.60 for domain and topic corrections and 0.35 for other implemented corrections.
|
|
191
|
-
These priors have full support and fixed finite precision.
|
|
192
|
-
|
|
193
|
-
Quality predictors include criteria, missingness indicators, duration, platform, and source indicators.
|
|
194
|
-
Several tags form one predictor vector for one judgment.
|
|
195
|
-
They do not create duplicate observations.
|
|
196
|
-
The engine rebuilds models from current training labels.
|
|
197
|
-
An edit replaces one effective observation.
|
|
198
|
-
|
|
199
|
-
The current interface uses four independent structured ratings.
|
|
200
|
-
Binary upvote and downvote commands and migration tools are not available.
|
|
201
|
-
Model version 8 retains its portable parameter groups for exact preference transfer.
|
|
202
|
-
|
|
203
|
-
## 5. Full ranking
|
|
204
|
-
|
|
205
|
-
The initial weighted score is
|
|
206
|
-
|
|
207
|
-
\[
|
|
208
|
-
B=0.60Q+0.25I+0.05E+0.05R+0.05D.
|
|
209
|
-
\]
|
|
210
|
-
|
|
211
|
-
All components use a scale from zero to 100.
|
|
212
|
-
Without structured feedback, quality and interest use their initial priors.
|
|
213
|
-
The next section defines the structured replacements.
|
|
214
|
-
\(E\) is community response.
|
|
215
|
-
For age \(a\) in years, recency is \(R=100/(1+a/10)\).
|
|
216
|
-
A missing publication date gives \(R=50\).
|
|
217
|
-
|
|
218
|
-
Creator diversity is \(D=100/(1+n+0.1h)\).
|
|
219
|
-
Here, \(n\) counts this creator in the developing list.
|
|
220
|
-
The value \(h\) counts the creator in recent recommendation exposures.
|
|
221
|
-
Randomness enters discovery rather than adding a random final-score bonus.
|
|
222
|
-
This score component measures creator diversity.
|
|
223
|
-
Screening policy 5 also applies a separate diversity rule when it constructs each platform list.
|
|
224
|
-
For each remaining candidate, count earlier picks that share a retrieval query, primary narrow topic, or secondary-school course format.
|
|
225
|
-
Choose the lowest count first, then the highest full score.
|
|
226
|
-
Primary labels match after normalization, including possessives and multiword phrase containment.
|
|
227
|
-
Broad subject labels do not create a match.
|
|
228
|
-
High-school and Gaokao markers come from titles and metadata tags.
|
|
229
|
-
Missing labels do not create a match.
|
|
230
|
-
A positive overlap count means the rule relaxed to fill the list from the assessed pool.
|
|
231
|
-
The saved `selection_diversity` field records that case.
|
|
232
|
-
This selection rule changes no quality grade, component weight, or preference posterior.
|
|
233
|
-
It adds no model calls. It cannot guarantee semantic uniqueness when labels or query coverage are incomplete.
|
|
234
|
-
Different videos about the same subject can therefore recur.
|
|
235
|
-
A subject match alone does not establish repeated content or equal explanatory depth.
|
|
236
|
-
|
|
237
|
-
Current profiles rank with \(B\), using learned structured quality and interest.
|
|
238
|
-
The initial component weights remain a ranking policy.
|
|
239
|
-
An empty internal overall model does not change this score.
|
|
240
|
-
Historical snapshots can retain an older overall adjustment.
|
|
241
|
-
Published historical lists retain their saved scores.
|
|
242
|
-
Later runs use the current model.
|
|
243
|
-
|
|
244
|
-
## 6. Structured feedback
|
|
245
|
-
|
|
246
|
-
Topic interest, broad-subject interest, depth fit, and video quality are independent dimensions.
|
|
247
|
-
Low, Medium, and High represent observed categories.
|
|
248
|
-
Blank means no observation.
|
|
249
|
-
Medium interest means indifference.
|
|
250
|
-
Medium quality means acceptable quality.
|
|
251
|
-
An empty field is not a Medium response.
|
|
252
|
-
The dashboard shows Not rated yet as a placeholder.
|
|
253
|
-
Use Clear to remove a saved dimension.
|
|
254
|
-
|
|
255
|
-
New assessments provide cited content topics and audience levels.
|
|
256
|
-
A separate bounded labeling job can update older videos from retained captions.
|
|
257
|
-
Search queries do not populate rating topics.
|
|
258
|
-
The user can edit each label or audience level.
|
|
259
|
-
|
|
260
|
-
Ratings provide the observations used for learning.
|
|
261
|
-
Tag exclusions remain separate explicit rules.
|
|
262
|
-
Applied tag rules operate during screening and final selection.
|
|
263
|
-
They require matching observed tags.
|
|
264
|
-
The user can remove an exclusion.
|
|
265
|
-
|
|
266
|
-
### Topic and broad-subject interest
|
|
267
|
-
|
|
268
|
-
Each user-selected topic or subject uses a separate Dirichlet categorical model.
|
|
269
|
-
For initial mean \(b\), its Low/Medium/High prior is
|
|
270
|
-
|
|
271
|
-
\[
|
|
272
|
-
\alpha=2\left((1-b)^2,\ 2b(1-b),\ b^2\right).
|
|
273
|
-
\]
|
|
274
|
-
|
|
275
|
-
Use \(b=0.75\) for an onboarding-related topic or subject and \(b=0.35\) otherwise.
|
|
276
|
-
Current distinct-video category counts \(n\) give posterior \(\operatorname{Dirichlet}(\alpha+n)\).
|
|
277
|
-
The interest index is \(100[P(\mathrm{High})+0.5P(\mathrm{Medium})]\).
|
|
278
|
-
|
|
279
|
-
The earliest explicit topic or depth action anchors the topic's parent domains.
|
|
280
|
-
Edits and clears preserve that anchor.
|
|
281
|
-
An explicit onboarding change can change the prior.
|
|
282
|
-
A new dislike cannot move the prior by changing the parent association.
|
|
283
|
-
|
|
284
|
-
A topic judgment does not update parent, sibling, global, or creator dislikes.
|
|
285
|
-
A matching specific-topic posterior takes precedence over a broad-subject posterior.
|
|
286
|
-
Several matching topic scores use an average.
|
|
287
|
-
Custom topic labels transfer only to matching normalized labels.
|
|
288
|
-
|
|
289
|
-
### Depth fit
|
|
290
|
-
|
|
291
|
-
Depth uses an exact categorical model for each observed topic and audience level.
|
|
292
|
-
The responses are want deeper, suitable, and want lighter.
|
|
293
|
-
Their Dirichlet prior is \((0.5,1,0.5)\).
|
|
294
|
-
For an observed level, \(s=P(\mathrm{suitable})\) multiplies interest odds by \(s/(1-s)\).
|
|
295
|
-
Its prior mean of 0.5 is neutral.
|
|
296
|
-
|
|
297
|
-
For a level without direct feedback, a separate directional model transfers information within that topic.
|
|
298
|
-
Encode introductory, intermediate, and advanced as \(l=0,1,2\).
|
|
299
|
-
A latent preferred level has prior \(\mu\sim N(1,1.5^2)\).
|
|
300
|
-
With \(w=0.75\), its likelihood is
|
|
301
|
-
|
|
302
|
-
\[
|
|
303
|
-
P(\mathrm{deeper}\mid\mu,l)=\sigma(\mu-l-w),
|
|
304
|
-
\quad P(\mathrm{lighter}\mid\mu,l)=\sigma(l-\mu-w),
|
|
305
|
-
\]
|
|
306
|
-
\[
|
|
307
|
-
P(\mathrm{suitable}\mid\mu,l)=1-P(\mathrm{deeper})-P(\mathrm{lighter}).
|
|
308
|
-
\]
|
|
309
|
-
|
|
310
|
-
Adaptive one-dimensional quadrature approximates this posterior.
|
|
311
|
-
The transfer adds \(\operatorname{logit}(s_{post})-\operatorname{logit}(s_{prior})\) to interest log odds.
|
|
312
|
-
Thus, no feedback gives no adjustment.
|
|
313
|
-
Want deeper shifts preference upward, while want lighter shifts it downward.
|
|
314
|
-
Direct categorical feedback takes precedence when that candidate level receives observations.
|
|
315
|
-
The directional model does not add duplicate categorical counts.
|
|
316
|
-
|
|
317
|
-
This transfer is a ranking policy, not a calibrated joint probability model.
|
|
318
|
-
Its fixed noise width cannot represent every possible response distribution.
|
|
319
|
-
The exact categorical consistency result applies to directly observed topic-level groups.
|
|
320
|
-
No unconditional convergence claim applies to transferred scores at an unobserved level.
|
|
321
|
-
|
|
322
|
-
The engine clips depth suitability to \(10^{-12}\) through \(1-10^{-12}\) before conversion to log odds.
|
|
323
|
-
Depth does not change quality or domain interest.
|
|
324
|
-
Unknown audience levels retain feedback but do not adjust future scores.
|
|
325
|
-
The model supplies a cited level when the evidence supports one.
|
|
326
|
-
The user can override that level.
|
|
327
|
-
The engine distinguishes an AI estimate from an explicit user level.
|
|
328
|
-
An explicit Unknown choice prevents the AI estimate from overriding that judgment.
|
|
329
|
-
Later labels can resolve a pending AI level.
|
|
330
|
-
Errors in estimated levels can still bias depth predictions.
|
|
331
|
-
Duration does not establish audience level.
|
|
332
|
-
|
|
333
|
-
### Ordinal quality
|
|
334
|
-
|
|
335
|
-
The quality model uses two logistic stages:
|
|
336
|
-
|
|
337
|
-
\[
|
|
338
|
-
a=P(\mathrm{Medium\ or\ High}),\quad b=P(\mathrm{High}\mid\mathrm{not\ Low}).
|
|
339
|
-
\]
|
|
340
|
-
|
|
341
|
-
The category probabilities are \((1-a,\ a(1-b),\ ab)\).
|
|
342
|
-
A Low response supplies only the first-stage event.
|
|
343
|
-
Medium and High responses also supply the conditional second-stage event.
|
|
344
|
-
This is one categorical likelihood, not duplicate votes or fractional Bernoulli labels.
|
|
345
|
-
|
|
346
|
-
The quality index is \(Q=100[0.5P(\mathrm{Medium})+P(\mathrm{High})]\).
|
|
347
|
-
It represents expected ordinal utility, not a binary approval percentage.
|
|
348
|
-
Initial stage offsets use the quality prior when the retired binary model has no observations.
|
|
349
|
-
Learned corrections can cancel these offsets.
|
|
350
|
-
An empty compatibility model supplies the baseline without retaining binary votes.
|
|
351
|
-
|
|
352
|
-
## 7. Calibration and evaluation
|
|
353
|
-
|
|
354
|
-
The calibration cohort contains thirty previously unrated videos, fifteen from each platform.
|
|
355
|
-
Selection balances available groups and limits repeated creators and retrieval families.
|
|
356
|
-
It excludes creators already present in active training feedback.
|
|
357
|
-
This does not guarantee semantic diversity across all possible subjects.
|
|
358
|
-
|
|
359
|
-
Known creators stay entirely in training or holdout.
|
|
360
|
-
Feedback from other videos by a reserved holdout creator also waits for release.
|
|
361
|
-
The engine saves evaluation before it releases reserved labels into training.
|
|
362
|
-
It releases a completed review with at least 26 judgments in each main channel.
|
|
363
|
-
An incomplete reservation also expires after sixty distinct newly rated videos.
|
|
364
|
-
Expiration does not establish successful calibration.
|
|
365
|
-
|
|
366
|
-
Reports include Brier score, log loss, prior comparisons, and creator-group bootstrap intervals.
|
|
367
|
-
Binary reliability bins compare predicted and observed response frequencies.
|
|
368
|
-
Categorical responses use multiclass losses.
|
|
369
|
-
The small-sample bootstrap does not guarantee nominal interval coverage.
|
|
370
|
-
|
|
371
|
-
The development check requires:
|
|
372
|
-
|
|
373
|
-
1. A thirty-video cohort and at least 26 judgments per main channel.
|
|
374
|
-
2. At least ten judgments per channel and platform.
|
|
375
|
-
3. At least eighteen training judgments for each active main response type.
|
|
376
|
-
4. At least eight holdout judgments with two examples in each of two response categories.
|
|
377
|
-
5. At least six holdout creator groups for each checked response type.
|
|
378
|
-
6. No detected Brier degradation relative to the prior.
|
|
379
|
-
|
|
380
|
-
The check includes mixed binary and structured response types separately.
|
|
381
|
-
Only known-level depth feedback can establish known-level depth coverage.
|
|
382
|
-
The separate overall model is outside this main-channel check.
|
|
383
|
-
Wide intervals do not prove equivalence or improvement.
|
|
384
|
-
|
|
385
|
-
Read-only status does not fit a missing model.
|
|
386
|
-
Changed labels or effective features invalidate affected frozen evaluations.
|
|
387
|
-
A new response type requires new holdout coverage.
|
|
388
|
-
Training on released labels does not replace the earlier test result.
|
|
389
|
-
|
|
390
|
-
Evidence maintenance adds derived feature records after source checks.
|
|
391
|
-
They preserve original ratings, scorecards, and captured features.
|
|
392
|
-
Incomplete source metadata can pause quality fitting.
|
|
393
|
-
Maintenance uses retained frames and captions without downloading replacement media.
|
|
394
|
-
|
|
395
|
-
## 8. Transfer of learned parameters
|
|
396
|
-
|
|
397
|
-
Preference format 4 transfers interests, model version 8 parameters, watched video IDs, a bounded exploration summary, and manual creator exclusions.
|
|
398
|
-
The engine rejects other preference formats before import changes the profile.
|
|
399
|
-
The included starter preset provides sanitized interests, parameters, and parameter uncertainty from the owner's export.
|
|
400
|
-
The preset excludes watched IDs, individual history, topic coverage, and creator exclusions.
|
|
401
|
-
Import replaces manual creator lists independently of learned parameters. These rules never create negative ratings.
|
|
402
|
-
Blacklist sync combines the account-specific blacklist with the current manual rules.
|
|
403
|
-
The exploration summary contains at most 1,024 hierarchy nodes, aggregate counts, and relative round ages.
|
|
404
|
-
It contains no individual video history.
|
|
405
|
-
Logistic channels retain parameter names, posterior means, and the full covariance or precision matrix.
|
|
406
|
-
Sparse fits aggregate their precision matrix before export. They do not export per-observation feature rows.
|
|
407
|
-
Categorical groups retain Dirichlet concentration parameters and aggregate observation counts.
|
|
408
|
-
Depth models retain their posterior summaries and a three-by-three table of level/response counts per topic.
|
|
409
|
-
These counts are sufficient statistics for the implemented depth likelihood. They contain no video identities or chronology.
|
|
410
|
-
The watched-video ID list contains no titles, ratings, or recommendation records.
|
|
411
|
-
The exploration summary preserves portable topic coverage without video identities or per-video history.
|
|
412
|
-
Import merges those IDs into the target profile's watched state to prevent exact repeats.
|
|
413
|
-
The target profile keeps its local ratings, video lists, tag rules, and account-specific blacklist copies.
|
|
414
|
-
Quality calibration retains both continuation-stage models.
|
|
415
|
-
Transfers exclude per-video topic and level corrections, test cohorts, and validation claims.
|
|
416
|
-
Local saved corrections remain visible in the dashboard.
|
|
417
|
-
They continue to label local videos without adding their old ratings to training.
|
|
418
|
-
|
|
419
|
-
For a logistic parameter vector, import uses the saved Gaussian approximation as a proper prior:
|
|
420
|
-
|
|
421
|
-
\[
|
|
422
|
-
\beta\sim N(m_0,\Sigma_0),\qquad
|
|
423
|
-
\log p(\beta\mid D_{\mathrm{new}})
|
|
424
|
-
=-\tfrac12(\beta-m_0)^\top\Sigma_0^{-1}(\beta-m_0)
|
|
425
|
-
+\sum_{i\in D_{\mathrm{new}}}\log p(y_i\mid x_i,\beta)+C.
|
|
426
|
-
\]
|
|
427
|
-
|
|
428
|
-
The implementation preserves off-diagonal correlations.
|
|
429
|
-
New coefficients receive the usual independent proper priors.
|
|
430
|
-
MAP fitting and a full Hessian update the imported Gaussian approximation.
|
|
431
|
-
This is sequential approximate Bayesian learning. It does not reconstruct the original non-Gaussian posterior from raw observations.
|
|
432
|
-
Immediately after import, predictions and parameter uncertainty match the exported approximation, subject to floating-point arithmetic.
|
|
433
|
-
Per-video corrections and local exclusion rules are outside this comparison.
|
|
434
|
-
|
|
435
|
-
For each categorical group, new counts add to the imported concentration vector.
|
|
436
|
-
Depth fitting combines imported aggregate counts with new counts and repeats its numerical integration.
|
|
437
|
-
Each topic uses at most nine level-response counts. This is the same likelihood as separate response rows.
|
|
438
|
-
Local ratings saved before import do not enter these likelihoods again.
|
|
439
|
-
Editing an older rating supplies one new post-import observation. Later edits replace that post-import observation.
|
|
440
|
-
Clearing it removes its post-import contribution.
|
|
441
|
-
Clearing cannot remove an unidentified historical observation embedded in the imported aggregate posterior.
|
|
442
|
-
Replacing the imported baseline with an earlier recovery file can restore its earlier parameters.
|
|
443
|
-
Repeated import replaces the baseline. It does not add two copies of the same parameters.
|
|
444
|
-
Frozen evaluation results cannot cross an import boundary.
|
|
445
|
-
|
|
446
|
-
The consistency assumptions below apply to a fixed imported proper prior and sufficiently many subsequent distinct-video observations.
|
|
447
|
-
Repeated transfers of successively approximated posteriors do not establish unconditional consistency.
|
|
448
|
-
|
|
449
|
-
## 9. Consistency assumptions
|
|
450
|
-
|
|
451
|
-
Infinite feedback alone is insufficient.
|
|
452
|
-
Repeated edits to one rating remain one observation.
|
|
453
|
-
Unobserved topics and confounded creator effects cannot become identifiable through unrelated feedback.
|
|
454
|
-
|
|
455
|
-
For a fixed model version and finite covered feature support, sufficient conditions are:
|
|
456
|
-
|
|
457
|
-
1. Distinct videos supply stationary, conditionally independent responses with probabilities strictly between zero and one.
|
|
458
|
-
2. Each covered context has positive limiting observation frequency. Missing labels do not select outcomes after conditioning on modeled features.
|
|
459
|
-
3. The true conditional logits lie within the implemented offset and feature family.
|
|
460
|
-
4. The prediction target is identifiable on the covered support. Redundant coefficients need not be separately identifiable.
|
|
461
|
-
5. The prior is proper and positive near the true target, with fixed finite precision.
|
|
462
|
-
6. Plug-in component offsets converge uniformly where an overall or ordinal stage uses them.
|
|
463
|
-
|
|
464
|
-
Under regular finite-dimensional conditions, the exact posterior concentrates around the true identified target.
|
|
465
|
-
Consequently, predictions converge on the covered support.
|
|
466
|
-
|
|
467
|
-
The normalized likelihood contrast is a negative weighted sum of Bernoulli KL divergences.
|
|
468
|
-
Positive coverage and identifiability separate the truth from alternatives.
|
|
469
|
-
The fixed Gaussian log prior divided by sample size tends to zero.
|
|
470
|
-
Its positive local density supplies prior mass near the truth.
|
|
471
|
-
The standard concentration argument then applies on the identifiable design span.
|
|
472
|
-
Null-space coefficients can retain prior uncertainty.
|
|
473
|
-
Uniformly vanishing offset errors preserve the limiting likelihood contrast.
|
|
474
|
-
|
|
475
|
-
Routine inference uses MAP with a Laplace approximation.
|
|
476
|
-
Small models use a full Hessian.
|
|
477
|
-
Larger models use matrix-free preconditioned conjugate gradients.
|
|
478
|
-
Prediction uses finite Gaussian quadrature.
|
|
479
|
-
These are numerical approximations, with conditional asymptotic justification rather than exact finite-sample posterior calculations.
|
|
480
|
-
|
|
481
|
-
For a fixed categorical group, the exact posterior is \(\operatorname{Dirichlet}(\alpha+n)\).
|
|
482
|
-
If that group's distinct-video sample grows under stationary independent responses, empirical proportions converge and posterior variance tends to zero.
|
|
483
|
-
Each topic or level group needs its own coverage.
|
|
484
|
-
The ordinal model also requires positive second-stage frequency and well-specified conditional models.
|
|
485
|
-
|
|
486
|
-
The project does not claim unconditional consistency for growing hierarchies, changing preferences, missing predictors, selective feedback, or arbitrary adaptive retrieval.
|
|
487
|
-
Difficulty-related abstentions can violate the missing-label assumption.
|
|
488
|
-
Model misspecification can produce a best-fitting approximation instead of the actual preference.
|
|
489
|
-
|
|
490
|
-
There is no fixed observation cap, forgetting window, or learned correction cap.
|
|
491
|
-
Unseen features retain prior uncertainty.
|
|
492
|
-
Evidence-sensitivity intervals remain separate from point predictions.
|
|
493
|
-
Persistent artificial measurement noise does not enter the point prediction.
|
|
494
|
-
|
|
495
|
-
## References
|
|
496
|
-
|
|
497
|
-
- [Stan regression models](https://mc-stan.org/docs/stan-users-guide/regression.html).
|
|
498
|
-
- [Stan categorical and Dirichlet distributions](https://mc-stan.org/docs/functions-reference/multinomial-distribution.html).
|
|
499
|
-
- [Miller, 2021](https://www.jmlr.org/papers/v22/20-469.html): posterior concentration and Laplace conditions.
|
|
500
|
-
- [Kasprzak, Giordano, and Broderick, 2025](https://www.jmlr.org/beta/papers/v26/24-0619.html): finite-sample Laplace accuracy.
|
|
501
|
-
- [Li et al.](https://arxiv.org/abs/1003.5956): assumptions for offline recommendation evaluation.
|
|
1
|
+
# Scoring, calibration, and mathematics
|
|
2
|
+
|
|
3
|
+
Broader Vids learns conditional preferences from explicit feedback.
|
|
4
|
+
It does not estimate an objective quality score for every video.
|
|
5
|
+
Its conclusions apply to the retrieved videos and feature contexts that receive feedback.
|
|
6
|
+
|
|
7
|
+
## Terms
|
|
8
|
+
|
|
9
|
+
| Term | Meaning |
|
|
10
|
+
| --- | --- |
|
|
11
|
+
| Prior | A model assumption before the current observations |
|
|
12
|
+
| Posterior | The distribution after the model includes those observations |
|
|
13
|
+
| Predictor | A measured or derived input to a model |
|
|
14
|
+
| Likelihood | The probability model for observed feedback |
|
|
15
|
+
| Holdout | Feedback reserved for testing before training |
|
|
16
|
+
| Calibration | Comparison between predicted responses and observed responses |
|
|
17
|
+
| Provenance | Information about a video's source and the uploader's contribution |
|
|
18
|
+
| Ordinal response | An ordered category such as Low, Medium, or High |
|
|
19
|
+
|
|
20
|
+
## 1. Eligibility
|
|
21
|
+
|
|
22
|
+
Eligibility checks operate separately from learned preferences.
|
|
23
|
+
The engine requires a verified playable duration greater than sixty seconds and at most ninety minutes.
|
|
24
|
+
The saved evidence quality must reach 60 points out of 100.
|
|
25
|
+
For new assessments without a usable transcript, subtract ten points from quality after adding the duration prior, with a floor of zero.
|
|
26
|
+
The same deduction applies to the learned quality component during ranking.
|
|
27
|
+
Audio quality is not a criterion.
|
|
28
|
+
Captions and frames support the four existing content criteria.
|
|
29
|
+
Only rubric-5 evidence is eligible for reuse or new recommendation work.
|
|
30
|
+
Unsupported evidence remains unchanged. It requires a new evidence run.
|
|
31
|
+
By default, the exact Bilibili tag `视频翻译` excludes a video from final selection.
|
|
32
|
+
Watched and dismissed videos cannot enter a new final list.
|
|
33
|
+
|
|
34
|
+
A learned preference cannot override these checks.
|
|
35
|
+
An unavailable video remains an access failure.
|
|
36
|
+
It does not receive an invented low-quality label.
|
|
37
|
+
|
|
38
|
+
## 2. Community response and screening
|
|
39
|
+
|
|
40
|
+
YouTube uses likes divided by views.
|
|
41
|
+
Bilibili uses likes, coins, and favorites divided by views.
|
|
42
|
+
These measures belong to community response, separate from content quality.
|
|
43
|
+
Before admission to the 100-video buffer, both platforms require views >= 100, likes >= 3, and likes/views >= 0.01.
|
|
44
|
+
Bilibili additionally requires coins/views >= 0.001 and favorites/views >= 0.002.
|
|
45
|
+
Missing required metrics fail admission.
|
|
46
|
+
These hard floors use raw rates before the smoothing described below.
|
|
47
|
+
|
|
48
|
+
For metric count \(c\), views \(v\), and platform baseline \(b\), the adjusted rate is
|
|
49
|
+
|
|
50
|
+
\[
|
|
51
|
+
\widehat r=\frac{c+1000b}{v+1000}.
|
|
52
|
+
\]
|
|
53
|
+
|
|
54
|
+
The baseline is the middle sorted rate when at least ten valid platform observations exist.
|
|
55
|
+
Otherwise, initial baselines are 0.03 for likes, 0.01 for coins, and 0.02 for favorites.
|
|
56
|
+
The implementation limits the baseline below by \(10^{-6}\).
|
|
57
|
+
|
|
58
|
+
The metric score is
|
|
59
|
+
|
|
60
|
+
\[
|
|
61
|
+
e=100\sigma\left(\operatorname{clip}\left(\log\frac{\max(\widehat r,10^{-12})}{b},-8,8\right)\right),
|
|
62
|
+
\qquad \sigma(z)=\frac{1}{1+e^{-z}}.
|
|
63
|
+
\]
|
|
64
|
+
|
|
65
|
+
The rate-score helper assigns 50 to a missing count or zero denominator.
|
|
66
|
+
Such entries fail the hard admission floors and cannot enter a new recommendation shortlist.
|
|
67
|
+
Bilibili averages its three metric scores with equal weights.
|
|
68
|
+
These correlated metrics do not count as independent evidence of quality.
|
|
69
|
+
|
|
70
|
+
Initial screening uses 70% community response and 30% title-anchor support.
|
|
71
|
+
An anchor is a query term found in the returned metadata.
|
|
72
|
+
Collection scans batches of up to 100 unique candidates.
|
|
73
|
+
The eligible pool targets 71 familiar and 29 exploratory retrievals.
|
|
74
|
+
Each batch fills only the remaining slots in its own group, using the highest screening scores.
|
|
75
|
+
Later batches target deficient groups and preserve source attribution through search continuations.
|
|
76
|
+
Duplicates and rejected candidates do not fill slots.
|
|
77
|
+
This balance applies to collection, before inspection selection.
|
|
78
|
+
Screening policy 5 combines 80% of the metadata screening score with 20% learned interest.
|
|
79
|
+
The metadata score retains its 70% community-response and 30% title-anchor weights.
|
|
80
|
+
The learned term uses topic-specific interest when verified title or tag context names that topic.
|
|
81
|
+
Otherwise, it uses a broad-subject estimate when available.
|
|
82
|
+
Unknown context receives a neutral value of 50.
|
|
83
|
+
Repeated mentions of one topic count once, including mentions shared by the title and tags.
|
|
84
|
+
Each topic also counts once when averaging verified depth estimates.
|
|
85
|
+
The screening model uses verified video context and does not infer quality.
|
|
86
|
+
It uses audience depth only when evidence or the user verifies the video level.
|
|
87
|
+
It does not use speculative quality estimates.
|
|
88
|
+
Screening policy 5 retains the 71 familiar and 29 exploratory eligible-pool targets.
|
|
89
|
+
It also retains thirty-five inspection slots and balances query coverage.
|
|
90
|
+
Inspection selection balances query coverage across thirty-five slots.
|
|
91
|
+
Each slot favors the candidate whose retrieval queries have the fewest existing slots.
|
|
92
|
+
For several query origins, the largest query count applies.
|
|
93
|
+
The applicable screening score and canonical ID resolve ties.
|
|
94
|
+
A missing query uses its retrieval family when available. Unknown sources share no invented group.
|
|
95
|
+
This rule permits repeats when fewer queries can supply candidates.
|
|
96
|
+
Pending entries compete on the same score without guaranteed slots.
|
|
97
|
+
A creator exclusion removes eligibility without creating negative feedback.
|
|
98
|
+
Creator IDs determine these exclusions.
|
|
99
|
+
Account names serve only as labels.
|
|
100
|
+
Unresolved creators remain eligible.
|
|
101
|
+
Each round retains a fixed creator-rule snapshot for screening, assessment scheduling, and final selection.
|
|
102
|
+
Rule changes apply to later rounds rather than changing an active or resumed round.
|
|
103
|
+
Manual and account-specific imported rules remain separate from learning parameters.
|
|
104
|
+
Policy 5 saves one learning-model reference when the round first executes.
|
|
105
|
+
Screening, resumed work, previews, and final selection use that reference.
|
|
106
|
+
Feedback saved during a round changes the model used by a later round.
|
|
107
|
+
Unsupported round policies cannot resume. Current rounds retain their saved settings.
|
|
108
|
+
|
|
109
|
+
## 3. Initial quality and interest
|
|
110
|
+
|
|
111
|
+
The assessment uses four content criteria:
|
|
112
|
+
|
|
113
|
+
| Criterion | Initial content weight | Meaning |
|
|
114
|
+
| --- | ---: | --- |
|
|
115
|
+
| Substance | 40% | Useful explanations or substantive content |
|
|
116
|
+
| Directness | 25% | Relevant content without avoidable repetition or filler |
|
|
117
|
+
| Original contribution | 25% | Evidence of the uploader's own contribution |
|
|
118
|
+
| Purposeful visuals | 10% | Visual material that supports the content |
|
|
119
|
+
|
|
120
|
+
Rubric 5 judges substance and sampled focus against the video's purpose.
|
|
121
|
+
Travel films can provide substance through concrete places, activities, and distinctive visual detail.
|
|
122
|
+
Crafts and games can provide substance through visible execution, development, or outcomes.
|
|
123
|
+
Instructional videos still need evidence of their explanations.
|
|
124
|
+
A software demonstration does not establish that it teaches the underlying algorithm.
|
|
125
|
+
Substance includes explanations needed to understand an instructional video.
|
|
126
|
+
For other video types, judge substantial content against the video's purpose.
|
|
127
|
+
A visual demonstration or performance does not need to teach a lesson.
|
|
128
|
+
Silent frames cannot establish musical performance quality.
|
|
129
|
+
Temporal gaps reduce confidence and widen the interval.
|
|
130
|
+
They do not automatically prevent assessment of sampled relevance and focus.
|
|
131
|
+
A title card alone or unrelated scenes can still leave the overall score unknown.
|
|
132
|
+
Unknown sound, factual accuracy, and full-video pacing remain explicit limitations.
|
|
133
|
+
This outcome is an assessment limitation, not a download failure or a low quality rating.
|
|
134
|
+
A course label alone does not reduce quality.
|
|
135
|
+
A short, complete explanation can have high quality.
|
|
136
|
+
Visual assessment includes text readability, useful contrast, framing, and legible diagrams.
|
|
137
|
+
Simple slides and phone recordings can satisfy these checks.
|
|
138
|
+
Caption evidence cannot establish visual quality or AI generation.
|
|
139
|
+
Creator reputation does not guarantee a high grade.
|
|
140
|
+
Existing scorecards remain stored. New work and cache reuse require rubric 5.
|
|
141
|
+
|
|
142
|
+
An assessed criterion has an integer grade from zero to four.
|
|
143
|
+
Its normalized value is \(u_j=\mathrm{grade}_j/4\).
|
|
144
|
+
An unknown criterion uses a prior mean of 0.5 and a separate missingness predictor.
|
|
145
|
+
Its stored grade remains unknown.
|
|
146
|
+
It does not inherit a high grade from another criterion.
|
|
147
|
+
|
|
148
|
+
For duration \(t\) seconds, the derived initial quality index is
|
|
149
|
+
|
|
150
|
+
\[
|
|
151
|
+
d=\min(t/3600,1),\qquad Q_0=95\sum_j w_j u_j+5d.
|
|
152
|
+
\]
|
|
153
|
+
|
|
154
|
+
Length contributes at most five points.
|
|
155
|
+
The contribution rises to one hour and then remains constant.
|
|
156
|
+
These initial weights express a design choice, not a fitted optimum.
|
|
157
|
+
The saved holistic evidence score remains a separate historical observation.
|
|
158
|
+
|
|
159
|
+
The initial interest probability \(I_0\) is 0.75 for a domain related to the user's stated interests.
|
|
160
|
+
It is 0.35 for a known unrelated domain and 0.50 when domains are unknown.
|
|
161
|
+
Search-domain labels describe retrieval context.
|
|
162
|
+
They do not prove the video's semantic topic.
|
|
163
|
+
|
|
164
|
+
New AI topic labels must be short subject names, with at most 120 characters and 12 words.
|
|
165
|
+
The validator rejects obvious citations, instructions, control characters, and editing notes in labels.
|
|
166
|
+
Invalid new output uses the existing bounded assessment fix workflow.
|
|
167
|
+
Read operations omit unusable labels from older scorecards and annotations.
|
|
168
|
+
They preserve raw evidence, numeric quality scores, saved ratings, and explicit user targets.
|
|
169
|
+
This check detects common formatting errors. It does not prove that a topic correctly describes the video.
|
|
170
|
+
|
|
171
|
+
Source links, repost declarations, `中配`, and YouDub disclosures inform originality assessment.
|
|
172
|
+
A source link alone does not prove copying.
|
|
173
|
+
An unfamiliar copyright enum remains uninterpreted.
|
|
174
|
+
The model must distinguish source teaching quality from the uploader's contribution.
|
|
175
|
+
|
|
176
|
+
## 4. Logistic components
|
|
177
|
+
|
|
178
|
+
Current ratings use categorical topic, subject, depth, and quality responses.
|
|
179
|
+
The quality model factors its categorical likelihood into logistic events.
|
|
180
|
+
Each event uses a predictor vector and a proper Gaussian prior:
|
|
181
|
+
|
|
182
|
+
\[
|
|
183
|
+
y_i\sim\operatorname{Bernoulli}(\sigma(o_i+x_i^T\beta)),
|
|
184
|
+
\qquad \beta_j\sim N(0,s_j^2).
|
|
185
|
+
\]
|
|
186
|
+
|
|
187
|
+
Numerical clipping keeps offsets finite.
|
|
188
|
+
It does not limit the final learned probability.
|
|
189
|
+
An offset also enters as a predictor, so learning can cancel that offset.
|
|
190
|
+
Prior standard deviations are 0.60 for domain and topic corrections and 0.35 for other implemented corrections.
|
|
191
|
+
These priors have full support and fixed finite precision.
|
|
192
|
+
|
|
193
|
+
Quality predictors include criteria, missingness indicators, duration, platform, and source indicators.
|
|
194
|
+
Several tags form one predictor vector for one judgment.
|
|
195
|
+
They do not create duplicate observations.
|
|
196
|
+
The engine rebuilds models from current training labels.
|
|
197
|
+
An edit replaces one effective observation.
|
|
198
|
+
|
|
199
|
+
The current interface uses four independent structured ratings.
|
|
200
|
+
Binary upvote and downvote commands and migration tools are not available.
|
|
201
|
+
Model version 8 retains its portable parameter groups for exact preference transfer.
|
|
202
|
+
|
|
203
|
+
## 5. Full ranking
|
|
204
|
+
|
|
205
|
+
The initial weighted score is
|
|
206
|
+
|
|
207
|
+
\[
|
|
208
|
+
B=0.60Q+0.25I+0.05E+0.05R+0.05D.
|
|
209
|
+
\]
|
|
210
|
+
|
|
211
|
+
All components use a scale from zero to 100.
|
|
212
|
+
Without structured feedback, quality and interest use their initial priors.
|
|
213
|
+
The next section defines the structured replacements.
|
|
214
|
+
\(E\) is community response.
|
|
215
|
+
For age \(a\) in years, recency is \(R=100/(1+a/10)\).
|
|
216
|
+
A missing publication date gives \(R=50\).
|
|
217
|
+
|
|
218
|
+
Creator diversity is \(D=100/(1+n+0.1h)\).
|
|
219
|
+
Here, \(n\) counts this creator in the developing list.
|
|
220
|
+
The value \(h\) counts the creator in recent recommendation exposures.
|
|
221
|
+
Randomness enters discovery rather than adding a random final-score bonus.
|
|
222
|
+
This score component measures creator diversity.
|
|
223
|
+
Screening policy 5 also applies a separate diversity rule when it constructs each platform list.
|
|
224
|
+
For each remaining candidate, count earlier picks that share a retrieval query, primary narrow topic, or secondary-school course format.
|
|
225
|
+
Choose the lowest count first, then the highest full score.
|
|
226
|
+
Primary labels match after normalization, including possessives and multiword phrase containment.
|
|
227
|
+
Broad subject labels do not create a match.
|
|
228
|
+
High-school and Gaokao markers come from titles and metadata tags.
|
|
229
|
+
Missing labels do not create a match.
|
|
230
|
+
A positive overlap count means the rule relaxed to fill the list from the assessed pool.
|
|
231
|
+
The saved `selection_diversity` field records that case.
|
|
232
|
+
This selection rule changes no quality grade, component weight, or preference posterior.
|
|
233
|
+
It adds no model calls. It cannot guarantee semantic uniqueness when labels or query coverage are incomplete.
|
|
234
|
+
Different videos about the same subject can therefore recur.
|
|
235
|
+
A subject match alone does not establish repeated content or equal explanatory depth.
|
|
236
|
+
|
|
237
|
+
Current profiles rank with \(B\), using learned structured quality and interest.
|
|
238
|
+
The initial component weights remain a ranking policy.
|
|
239
|
+
An empty internal overall model does not change this score.
|
|
240
|
+
Historical snapshots can retain an older overall adjustment.
|
|
241
|
+
Published historical lists retain their saved scores.
|
|
242
|
+
Later runs use the current model.
|
|
243
|
+
|
|
244
|
+
## 6. Structured feedback
|
|
245
|
+
|
|
246
|
+
Topic interest, broad-subject interest, depth fit, and video quality are independent dimensions.
|
|
247
|
+
Low, Medium, and High represent observed categories.
|
|
248
|
+
Blank means no observation.
|
|
249
|
+
Medium interest means indifference.
|
|
250
|
+
Medium quality means acceptable quality.
|
|
251
|
+
An empty field is not a Medium response.
|
|
252
|
+
The dashboard shows Not rated yet as a placeholder.
|
|
253
|
+
Use Clear to remove a saved dimension.
|
|
254
|
+
|
|
255
|
+
New assessments provide cited content topics and audience levels.
|
|
256
|
+
A separate bounded labeling job can update older videos from retained captions.
|
|
257
|
+
Search queries do not populate rating topics.
|
|
258
|
+
The user can edit each label or audience level.
|
|
259
|
+
|
|
260
|
+
Ratings provide the observations used for learning.
|
|
261
|
+
Tag exclusions remain separate explicit rules.
|
|
262
|
+
Applied tag rules operate during screening and final selection.
|
|
263
|
+
They require matching observed tags.
|
|
264
|
+
The user can remove an exclusion.
|
|
265
|
+
|
|
266
|
+
### Topic and broad-subject interest
|
|
267
|
+
|
|
268
|
+
Each user-selected topic or subject uses a separate Dirichlet categorical model.
|
|
269
|
+
For initial mean \(b\), its Low/Medium/High prior is
|
|
270
|
+
|
|
271
|
+
\[
|
|
272
|
+
\alpha=2\left((1-b)^2,\ 2b(1-b),\ b^2\right).
|
|
273
|
+
\]
|
|
274
|
+
|
|
275
|
+
Use \(b=0.75\) for an onboarding-related topic or subject and \(b=0.35\) otherwise.
|
|
276
|
+
Current distinct-video category counts \(n\) give posterior \(\operatorname{Dirichlet}(\alpha+n)\).
|
|
277
|
+
The interest index is \(100[P(\mathrm{High})+0.5P(\mathrm{Medium})]\).
|
|
278
|
+
|
|
279
|
+
The earliest explicit topic or depth action anchors the topic's parent domains.
|
|
280
|
+
Edits and clears preserve that anchor.
|
|
281
|
+
An explicit onboarding change can change the prior.
|
|
282
|
+
A new dislike cannot move the prior by changing the parent association.
|
|
283
|
+
|
|
284
|
+
A topic judgment does not update parent, sibling, global, or creator dislikes.
|
|
285
|
+
A matching specific-topic posterior takes precedence over a broad-subject posterior.
|
|
286
|
+
Several matching topic scores use an average.
|
|
287
|
+
Custom topic labels transfer only to matching normalized labels.
|
|
288
|
+
|
|
289
|
+
### Depth fit
|
|
290
|
+
|
|
291
|
+
Depth uses an exact categorical model for each observed topic and audience level.
|
|
292
|
+
The responses are want deeper, suitable, and want lighter.
|
|
293
|
+
Their Dirichlet prior is \((0.5,1,0.5)\).
|
|
294
|
+
For an observed level, \(s=P(\mathrm{suitable})\) multiplies interest odds by \(s/(1-s)\).
|
|
295
|
+
Its prior mean of 0.5 is neutral.
|
|
296
|
+
|
|
297
|
+
For a level without direct feedback, a separate directional model transfers information within that topic.
|
|
298
|
+
Encode introductory, intermediate, and advanced as \(l=0,1,2\).
|
|
299
|
+
A latent preferred level has prior \(\mu\sim N(1,1.5^2)\).
|
|
300
|
+
With \(w=0.75\), its likelihood is
|
|
301
|
+
|
|
302
|
+
\[
|
|
303
|
+
P(\mathrm{deeper}\mid\mu,l)=\sigma(\mu-l-w),
|
|
304
|
+
\quad P(\mathrm{lighter}\mid\mu,l)=\sigma(l-\mu-w),
|
|
305
|
+
\]
|
|
306
|
+
\[
|
|
307
|
+
P(\mathrm{suitable}\mid\mu,l)=1-P(\mathrm{deeper})-P(\mathrm{lighter}).
|
|
308
|
+
\]
|
|
309
|
+
|
|
310
|
+
Adaptive one-dimensional quadrature approximates this posterior.
|
|
311
|
+
The transfer adds \(\operatorname{logit}(s_{post})-\operatorname{logit}(s_{prior})\) to interest log odds.
|
|
312
|
+
Thus, no feedback gives no adjustment.
|
|
313
|
+
Want deeper shifts preference upward, while want lighter shifts it downward.
|
|
314
|
+
Direct categorical feedback takes precedence when that candidate level receives observations.
|
|
315
|
+
The directional model does not add duplicate categorical counts.
|
|
316
|
+
|
|
317
|
+
This transfer is a ranking policy, not a calibrated joint probability model.
|
|
318
|
+
Its fixed noise width cannot represent every possible response distribution.
|
|
319
|
+
The exact categorical consistency result applies to directly observed topic-level groups.
|
|
320
|
+
No unconditional convergence claim applies to transferred scores at an unobserved level.
|
|
321
|
+
|
|
322
|
+
The engine clips depth suitability to \(10^{-12}\) through \(1-10^{-12}\) before conversion to log odds.
|
|
323
|
+
Depth does not change quality or domain interest.
|
|
324
|
+
Unknown audience levels retain feedback but do not adjust future scores.
|
|
325
|
+
The model supplies a cited level when the evidence supports one.
|
|
326
|
+
The user can override that level.
|
|
327
|
+
The engine distinguishes an AI estimate from an explicit user level.
|
|
328
|
+
An explicit Unknown choice prevents the AI estimate from overriding that judgment.
|
|
329
|
+
Later labels can resolve a pending AI level.
|
|
330
|
+
Errors in estimated levels can still bias depth predictions.
|
|
331
|
+
Duration does not establish audience level.
|
|
332
|
+
|
|
333
|
+
### Ordinal quality
|
|
334
|
+
|
|
335
|
+
The quality model uses two logistic stages:
|
|
336
|
+
|
|
337
|
+
\[
|
|
338
|
+
a=P(\mathrm{Medium\ or\ High}),\quad b=P(\mathrm{High}\mid\mathrm{not\ Low}).
|
|
339
|
+
\]
|
|
340
|
+
|
|
341
|
+
The category probabilities are \((1-a,\ a(1-b),\ ab)\).
|
|
342
|
+
A Low response supplies only the first-stage event.
|
|
343
|
+
Medium and High responses also supply the conditional second-stage event.
|
|
344
|
+
This is one categorical likelihood, not duplicate votes or fractional Bernoulli labels.
|
|
345
|
+
|
|
346
|
+
The quality index is \(Q=100[0.5P(\mathrm{Medium})+P(\mathrm{High})]\).
|
|
347
|
+
It represents expected ordinal utility, not a binary approval percentage.
|
|
348
|
+
Initial stage offsets use the quality prior when the retired binary model has no observations.
|
|
349
|
+
Learned corrections can cancel these offsets.
|
|
350
|
+
An empty compatibility model supplies the baseline without retaining binary votes.
|
|
351
|
+
|
|
352
|
+
## 7. Calibration and evaluation
|
|
353
|
+
|
|
354
|
+
The calibration cohort contains thirty previously unrated videos, fifteen from each platform.
|
|
355
|
+
Selection balances available groups and limits repeated creators and retrieval families.
|
|
356
|
+
It excludes creators already present in active training feedback.
|
|
357
|
+
This does not guarantee semantic diversity across all possible subjects.
|
|
358
|
+
|
|
359
|
+
Known creators stay entirely in training or holdout.
|
|
360
|
+
Feedback from other videos by a reserved holdout creator also waits for release.
|
|
361
|
+
The engine saves evaluation before it releases reserved labels into training.
|
|
362
|
+
It releases a completed review with at least 26 judgments in each main channel.
|
|
363
|
+
An incomplete reservation also expires after sixty distinct newly rated videos.
|
|
364
|
+
Expiration does not establish successful calibration.
|
|
365
|
+
|
|
366
|
+
Reports include Brier score, log loss, prior comparisons, and creator-group bootstrap intervals.
|
|
367
|
+
Binary reliability bins compare predicted and observed response frequencies.
|
|
368
|
+
Categorical responses use multiclass losses.
|
|
369
|
+
The small-sample bootstrap does not guarantee nominal interval coverage.
|
|
370
|
+
|
|
371
|
+
The development check requires:
|
|
372
|
+
|
|
373
|
+
1. A thirty-video cohort and at least 26 judgments per main channel.
|
|
374
|
+
2. At least ten judgments per channel and platform.
|
|
375
|
+
3. At least eighteen training judgments for each active main response type.
|
|
376
|
+
4. At least eight holdout judgments with two examples in each of two response categories.
|
|
377
|
+
5. At least six holdout creator groups for each checked response type.
|
|
378
|
+
6. No detected Brier degradation relative to the prior.
|
|
379
|
+
|
|
380
|
+
The check includes mixed binary and structured response types separately.
|
|
381
|
+
Only known-level depth feedback can establish known-level depth coverage.
|
|
382
|
+
The separate overall model is outside this main-channel check.
|
|
383
|
+
Wide intervals do not prove equivalence or improvement.
|
|
384
|
+
|
|
385
|
+
Read-only status does not fit a missing model.
|
|
386
|
+
Changed labels or effective features invalidate affected frozen evaluations.
|
|
387
|
+
A new response type requires new holdout coverage.
|
|
388
|
+
Training on released labels does not replace the earlier test result.
|
|
389
|
+
|
|
390
|
+
Evidence maintenance adds derived feature records after source checks.
|
|
391
|
+
They preserve original ratings, scorecards, and captured features.
|
|
392
|
+
Incomplete source metadata can pause quality fitting.
|
|
393
|
+
Maintenance uses retained frames and captions without downloading replacement media.
|
|
394
|
+
|
|
395
|
+
## 8. Transfer of learned parameters
|
|
396
|
+
|
|
397
|
+
Preference format 4 transfers interests, model version 8 parameters, watched video IDs, a bounded exploration summary, and manual creator exclusions.
|
|
398
|
+
The engine rejects other preference formats before import changes the profile.
|
|
399
|
+
The included starter preset provides sanitized interests, parameters, and parameter uncertainty from the owner's export.
|
|
400
|
+
The preset excludes watched IDs, individual history, topic coverage, and creator exclusions.
|
|
401
|
+
Import replaces manual creator lists independently of learned parameters. These rules never create negative ratings.
|
|
402
|
+
Blacklist sync combines the account-specific blacklist with the current manual rules.
|
|
403
|
+
The exploration summary contains at most 1,024 hierarchy nodes, aggregate counts, and relative round ages.
|
|
404
|
+
It contains no individual video history.
|
|
405
|
+
Logistic channels retain parameter names, posterior means, and the full covariance or precision matrix.
|
|
406
|
+
Sparse fits aggregate their precision matrix before export. They do not export per-observation feature rows.
|
|
407
|
+
Categorical groups retain Dirichlet concentration parameters and aggregate observation counts.
|
|
408
|
+
Depth models retain their posterior summaries and a three-by-three table of level/response counts per topic.
|
|
409
|
+
These counts are sufficient statistics for the implemented depth likelihood. They contain no video identities or chronology.
|
|
410
|
+
The watched-video ID list contains no titles, ratings, or recommendation records.
|
|
411
|
+
The exploration summary preserves portable topic coverage without video identities or per-video history.
|
|
412
|
+
Import merges those IDs into the target profile's watched state to prevent exact repeats.
|
|
413
|
+
The target profile keeps its local ratings, video lists, tag rules, and account-specific blacklist copies.
|
|
414
|
+
Quality calibration retains both continuation-stage models.
|
|
415
|
+
Transfers exclude per-video topic and level corrections, test cohorts, and validation claims.
|
|
416
|
+
Local saved corrections remain visible in the dashboard.
|
|
417
|
+
They continue to label local videos without adding their old ratings to training.
|
|
418
|
+
|
|
419
|
+
For a logistic parameter vector, import uses the saved Gaussian approximation as a proper prior:
|
|
420
|
+
|
|
421
|
+
\[
|
|
422
|
+
\beta\sim N(m_0,\Sigma_0),\qquad
|
|
423
|
+
\log p(\beta\mid D_{\mathrm{new}})
|
|
424
|
+
=-\tfrac12(\beta-m_0)^\top\Sigma_0^{-1}(\beta-m_0)
|
|
425
|
+
+\sum_{i\in D_{\mathrm{new}}}\log p(y_i\mid x_i,\beta)+C.
|
|
426
|
+
\]
|
|
427
|
+
|
|
428
|
+
The implementation preserves off-diagonal correlations.
|
|
429
|
+
New coefficients receive the usual independent proper priors.
|
|
430
|
+
MAP fitting and a full Hessian update the imported Gaussian approximation.
|
|
431
|
+
This is sequential approximate Bayesian learning. It does not reconstruct the original non-Gaussian posterior from raw observations.
|
|
432
|
+
Immediately after import, predictions and parameter uncertainty match the exported approximation, subject to floating-point arithmetic.
|
|
433
|
+
Per-video corrections and local exclusion rules are outside this comparison.
|
|
434
|
+
|
|
435
|
+
For each categorical group, new counts add to the imported concentration vector.
|
|
436
|
+
Depth fitting combines imported aggregate counts with new counts and repeats its numerical integration.
|
|
437
|
+
Each topic uses at most nine level-response counts. This is the same likelihood as separate response rows.
|
|
438
|
+
Local ratings saved before import do not enter these likelihoods again.
|
|
439
|
+
Editing an older rating supplies one new post-import observation. Later edits replace that post-import observation.
|
|
440
|
+
Clearing it removes its post-import contribution.
|
|
441
|
+
Clearing cannot remove an unidentified historical observation embedded in the imported aggregate posterior.
|
|
442
|
+
Replacing the imported baseline with an earlier recovery file can restore its earlier parameters.
|
|
443
|
+
Repeated import replaces the baseline. It does not add two copies of the same parameters.
|
|
444
|
+
Frozen evaluation results cannot cross an import boundary.
|
|
445
|
+
|
|
446
|
+
The consistency assumptions below apply to a fixed imported proper prior and sufficiently many subsequent distinct-video observations.
|
|
447
|
+
Repeated transfers of successively approximated posteriors do not establish unconditional consistency.
|
|
448
|
+
|
|
449
|
+
## 9. Consistency assumptions
|
|
450
|
+
|
|
451
|
+
Infinite feedback alone is insufficient.
|
|
452
|
+
Repeated edits to one rating remain one observation.
|
|
453
|
+
Unobserved topics and confounded creator effects cannot become identifiable through unrelated feedback.
|
|
454
|
+
|
|
455
|
+
For a fixed model version and finite covered feature support, sufficient conditions are:
|
|
456
|
+
|
|
457
|
+
1. Distinct videos supply stationary, conditionally independent responses with probabilities strictly between zero and one.
|
|
458
|
+
2. Each covered context has positive limiting observation frequency. Missing labels do not select outcomes after conditioning on modeled features.
|
|
459
|
+
3. The true conditional logits lie within the implemented offset and feature family.
|
|
460
|
+
4. The prediction target is identifiable on the covered support. Redundant coefficients need not be separately identifiable.
|
|
461
|
+
5. The prior is proper and positive near the true target, with fixed finite precision.
|
|
462
|
+
6. Plug-in component offsets converge uniformly where an overall or ordinal stage uses them.
|
|
463
|
+
|
|
464
|
+
Under regular finite-dimensional conditions, the exact posterior concentrates around the true identified target.
|
|
465
|
+
Consequently, predictions converge on the covered support.
|
|
466
|
+
|
|
467
|
+
The normalized likelihood contrast is a negative weighted sum of Bernoulli KL divergences.
|
|
468
|
+
Positive coverage and identifiability separate the truth from alternatives.
|
|
469
|
+
The fixed Gaussian log prior divided by sample size tends to zero.
|
|
470
|
+
Its positive local density supplies prior mass near the truth.
|
|
471
|
+
The standard concentration argument then applies on the identifiable design span.
|
|
472
|
+
Null-space coefficients can retain prior uncertainty.
|
|
473
|
+
Uniformly vanishing offset errors preserve the limiting likelihood contrast.
|
|
474
|
+
|
|
475
|
+
Routine inference uses MAP with a Laplace approximation.
|
|
476
|
+
Small models use a full Hessian.
|
|
477
|
+
Larger models use matrix-free preconditioned conjugate gradients.
|
|
478
|
+
Prediction uses finite Gaussian quadrature.
|
|
479
|
+
These are numerical approximations, with conditional asymptotic justification rather than exact finite-sample posterior calculations.
|
|
480
|
+
|
|
481
|
+
For a fixed categorical group, the exact posterior is \(\operatorname{Dirichlet}(\alpha+n)\).
|
|
482
|
+
If that group's distinct-video sample grows under stationary independent responses, empirical proportions converge and posterior variance tends to zero.
|
|
483
|
+
Each topic or level group needs its own coverage.
|
|
484
|
+
The ordinal model also requires positive second-stage frequency and well-specified conditional models.
|
|
485
|
+
|
|
486
|
+
The project does not claim unconditional consistency for growing hierarchies, changing preferences, missing predictors, selective feedback, or arbitrary adaptive retrieval.
|
|
487
|
+
Difficulty-related abstentions can violate the missing-label assumption.
|
|
488
|
+
Model misspecification can produce a best-fitting approximation instead of the actual preference.
|
|
489
|
+
|
|
490
|
+
There is no fixed observation cap, forgetting window, or learned correction cap.
|
|
491
|
+
Unseen features retain prior uncertainty.
|
|
492
|
+
Evidence-sensitivity intervals remain separate from point predictions.
|
|
493
|
+
Persistent artificial measurement noise does not enter the point prediction.
|
|
494
|
+
|
|
495
|
+
## References
|
|
496
|
+
|
|
497
|
+
- [Stan regression models](https://mc-stan.org/docs/stan-users-guide/regression.html).
|
|
498
|
+
- [Stan categorical and Dirichlet distributions](https://mc-stan.org/docs/functions-reference/multinomial-distribution.html).
|
|
499
|
+
- [Miller, 2021](https://www.jmlr.org/papers/v22/20-469.html): posterior concentration and Laplace conditions.
|
|
500
|
+
- [Kasprzak, Giordano, and Broderick, 2025](https://www.jmlr.org/beta/papers/v26/24-0619.html): finite-sample Laplace accuracy.
|
|
501
|
+
- [Li et al.](https://arxiv.org/abs/1003.5956): assumptions for offline recommendation evaluation.
|