@heripo/research-radar 6.0.3 → 6.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +29 -15
- package/dist/index.d.ts +192 -5
- package/dist/index.js +1700 -942
- package/dist/index.js.map +1 -1
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -23,7 +23,7 @@ Previously reported service metrics were $0.2–1 per issue and 15% CTR. These a
|
|
|
23
23
|
|
|
24
24
|
- Type-safe TypeScript with strict interfaces
|
|
25
25
|
- Provider pattern for swapping components (Crawling/Analysis/Content/Email)
|
|
26
|
-
-
|
|
26
|
+
- 73 active crawling targets across heritage agencies, museums, academic societies, filtered at runtime by robots.txt, plus newly published journal issues injected by the application
|
|
27
27
|
- OpenAI GPT-6 defaults for every LLM task, with per-task model overrides
|
|
28
28
|
- Built-in retries, chain options, preview emails
|
|
29
29
|
|
|
@@ -212,21 +212,30 @@ deadlines }` from a successful refresh skips extraction only if a fresh fetch
|
|
|
212
212
|
|
|
213
213
|
Pass host-selected verified actions to
|
|
214
214
|
`renderVerifiedDeadlineReminders(reminders, publicationDate)` and place its result
|
|
215
|
-
in `templateOptions.deadlineRemindersMarkdown`.
|
|
216
|
-
|
|
217
|
-
|
|
218
|
-
|
|
215
|
+
in `templateOptions.deadlineRemindersMarkdown`. Active deadlines/events from
|
|
216
|
+
D-3 through today take priority; ongoing events through a known end date fill
|
|
217
|
+
any remaining slots. Each group is ordered by date and time, with at most ten
|
|
218
|
+
entries overall (`DEADLINE_REMINDER_LEAD_DAYS`, `DEADLINE_REMINDER_LIMIT`). It preserves supplied periods and times
|
|
219
219
|
without inventing 23:59. The heading is **다가오는 일정·마감**. Today's entries say
|
|
220
220
|
**오늘 마감**, **오늘 개최** or **오늘 공개**, while ongoing events say **진행 중**.
|
|
221
221
|
An event's start time is not a cutoff: only its stated end time expires it during
|
|
222
222
|
the end date. Without an end date, it is listed only through its start date. The provider appends it to saved Markdown and HTML after
|
|
223
|
-
generation, so reminders add no
|
|
224
|
-
repeat suppression and publication history. Empty sections are omitted.
|
|
223
|
+
generation, so reminders add no article-generation context. The host owns
|
|
224
|
+
reminder scheduling, repeat suppression and publication history. Empty sections are omitted.
|
|
225
225
|
|
|
226
226
|
Prefer `templateOptions.verifiedDeadlineReminders` with freshly verified historical
|
|
227
227
|
actions. The provider merges these with source-grounded deadlines from articles
|
|
228
|
-
actually linked in the generated body
|
|
229
|
-
|
|
228
|
+
actually linked in the generated body and displays at most ten eligible
|
|
229
|
+
deadlines/events in one linked section. `generateNewsletter()` asks the
|
|
230
|
+
importance model to choose those rows from every eligible entry: it answers with
|
|
231
|
+
candidate indices only, favours actions readers can take, keeps D-1 to D-3
|
|
232
|
+
alongside today's entries and drops the same event announced by several
|
|
233
|
+
sources. Titles, dates and links are always rendered from the verified rows,
|
|
234
|
+
and an error, a 60-second timeout or an unusable answer falls back to the first
|
|
235
|
+
ten by date. A short selection is filled in date order, skipping the rows the
|
|
236
|
+
model named as duplicates. Set
|
|
237
|
+
`recentReminderCount` on a reminder to tell the selection how many issues in
|
|
238
|
+
the past week already listed it. Deduplication includes the
|
|
230
239
|
entry type, so a registration cutoff cannot replace an attendance schedule.
|
|
231
240
|
Event identity also includes the start/end dates and times, preserving separate
|
|
232
241
|
occurrences with the same title and kind. Deadline action deduplication is unchanged.
|
|
@@ -254,6 +263,7 @@ formats, omit it for the default, or pass an empty string to suppress it.
|
|
|
254
263
|
| `templateOptions` | Default or KRAS newsletter branding and Markdown sections (see below) |
|
|
255
264
|
| `promptProvider` | Core `PromptProvider` overriding the package's own LLM prompts entirely; omit to use the package's prompts, which fall back to core's defaults |
|
|
256
265
|
| `excavationReportSource` | Supplies 국가유산청 발굴조사 보고서 entries from the application; that board is then never crawled. Omit to crawl it as before |
|
|
266
|
+
| `journalIssueSource` | Supplies newly published journal issues (such as KCI) from the application, one article per issue. Omit and that target collects nothing |
|
|
257
267
|
| `previewNewsletter` | Fetch a saved `Newsletter` and send it through a supplied core `EmailService` |
|
|
258
268
|
|
|
259
269
|
`previewNewsletter` requires `fetchNewsletterForPreview: () => Promise<Newsletter>`, `emailService` (with `send(message)`), and `emailMessage` (core `EmailMessage` without `subject`, `html`, or `text`). The core fills those three fields and skips preview delivery when no newsletter was created. Ensure the callback fetches the issue saved by this run.
|
|
@@ -331,20 +341,24 @@ The optional `contentGeneration: { provider, apiKey, model? }` compatibility pat
|
|
|
331
341
|
|
|
332
342
|
| Group | Active | Commented out |
|
|
333
343
|
| ---------- | -----: | ------------: |
|
|
334
|
-
| News |
|
|
344
|
+
| News | 56 | 1 |
|
|
335
345
|
| Business | 5 | 0 |
|
|
336
346
|
| Employment | 12 | 0 |
|
|
337
|
-
| **Total** | **
|
|
347
|
+
| **Total** | **73** | **1** |
|
|
338
348
|
|
|
339
|
-
Only the excavation status board stays commented out, as low-value fragmented data. Boards that a site's robots.txt restricts are configured normally and refused at runtime by the robots.txt check, so the configuration does not have to track each site's policy by hand. With current policies 14 of the
|
|
349
|
+
Only the excavation status board stays commented out, as low-value fragmented data. Boards that a site's robots.txt restricts are configured normally and refused at runtime by the robots.txt check, so the configuration does not have to track each site's policy by hand. With current policies 14 of the 73 targets are refused.
|
|
340
350
|
|
|
341
351
|
Three targets are read from data.go.kr open APIs instead of scraped: 나라일터 (`PblJobService`) and 알리오 (`recruitment`) under Employment, and 나라장터 (`BidPublicInfoService`) under Business. Each is a single round of requests, and `publicDataApiKey` supplies the service key — omit it and they answer with an empty list without making a request. The boards carry every public-sector vacancy and procurement notice in the country, so both are narrowed before analysis. The job boards use `src/crawling/heritage-job-filter.ts`, whose exclusions are role nouns and hold up as patterns; roughly 2% of postings survive, about 2.6 a day. 나라장터 instead uses `src/crawling/heritage-triage.ts`, which asks a cheap model about 100 titles per request: heritage vocabulary cannot be settled by substring in Korean, and the deterministic filter remains behind it as the fallback when a batch fails.
|
|
342
352
|
|
|
353
|
+
Newly published journal issues are not crawled. The application supplies them through `journalIssueSource` — heripo-web reads them from its KCI paper catalog — and the `KCI_학술지_신간` target serves them from that source, so its URL is never requested. Each issue (권호) becomes one article: its `uniqId` is the issue's `externalId` and its URL the issue's `url`. `usedArticles` carry only the application's article id and URL, so the application records which issues a saved newsletter introduced through the `uniqId` it stored in `saveCrawledArticles`, or through the issue URL. The article body, which scoring sees, lists every paper with its authors, keywords and an abstract truncated to 300 characters, after the keywords several papers share. Without a source the target answers with an empty list. This replaces the former crawl of the 국립문화유산연구원 헤리티지:역사와 과학 and 보존과학연구 boards.
|
|
354
|
+
|
|
355
|
+
The generation model never writes about these issues. `ContentGenerateProvider` takes them out of its candidates and renders a `📖 새로 나온 학술지` section itself, placed before `📑 사업·입찰 공고` (or `📌 마무리` when there is no procurement section). Each issue links to its KCI table of contents and shows, as a table with their authors, the papers its readers would want to read, most interesting first; the rest are collapsed into a count and their topics. The criterion is the readers' interest, not academic discipline — a 역사학 paper can be left out and a non-archaeology one shown — and the readers are 한국고고학회 회원 in KRAS mode. The importance model only chooses and orders paper indices and names the topics of the papers it leaves out — titles, links and authors come from the injected data. If the selection fails or times out, every paper is listed; if the run cannot load the issues, the articles go to the model as ordinary candidates. The held-aside articles are appended to `usedArticles`, so they are recorded with the issue even though the model never saw them. As a result the title, briefing and 마무리 do not mention journals, and journal issues do not count toward the publication criteria.
|
|
356
|
+
|
|
343
357
|
Crawling fetches pass through that check (`src/crawling/robots.ts`) before reaching the network: a disallowed request is refused rather than sent, and a missing or unreachable robots.txt allows it. `robotsExemptOrigins` in `src/config/index.ts` lists origins exempted from the check, each with its reason.
|
|
344
358
|
|
|
345
359
|
Sources include the Korea Heritage Service, National Research Institute of Cultural Heritage, National Research Institute of Maritime Heritage, Korea Heritage Agency, Korea Association of Archaeological Heritage, archaeological societies, and national museums.
|
|
346
360
|
|
|
347
|
-
[src/parsers/](./src/parsers/) contains 23 organization-specific
|
|
361
|
+
[src/parsers/](./src/parsers/) contains 24 parser modules (23 organization-specific, plus the injected journal issue adapter) plus shared date and URL utilities. List parsers return `ParsedTargetListItem[]` (title, date, detail URL, date type, and optional source ID); detail parsers return Markdown `detailContent` and attachment/image flags. Parsers can be synchronous or asynchronous. KRAS, Yeongnam Archaeological Society, and maritime heritage sources use additional API requests for client-rendered content.
|
|
348
362
|
|
|
349
363
|
`CrawlingProvider` uses a maximum concurrency of 5 and wraps the supplied fetch to route KRAS public detail URLs to its detail API, while retaining public URLs in article metadata. When constructing your own pipeline, use the provider's fetch together with its target groups.
|
|
350
364
|
|
|
@@ -356,7 +370,7 @@ import { getSourceList } from '@heripo/research-radar';
|
|
|
356
370
|
const groups = getSourceList(); // [{ id, name, sources: [{ id, name, url }] }]
|
|
357
371
|
```
|
|
358
372
|
|
|
359
|
-
`createCrawlingTargetGroups(customFetch?)`, `getSourceList()`, `contentOptions`, `newsletterConfig`, `llmConfig`, and `
|
|
373
|
+
`createCrawlingTargetGroups(customFetch?)`, `getSourceList()`, `contentOptions`, `newsletterConfig`, `llmConfig`, `researchRadarPromptProvider`, and `JOURNAL_ISSUE_LIST_URL` (the journal issue target's URL, for finding the issue articles an application saved) are public exports. `ExcavationReport`, `ExcavationReportSource`, `JournalIssue`, `JournalIssuePaper`, and `JournalIssueSource` are exported as types. The package also exports the three provider classes, `DateService`, `TaskService`, and their public configuration/dependency types through [src/index.ts](./src/index.ts).
|
|
360
374
|
|
|
361
375
|
## Email templates
|
|
362
376
|
|
|
@@ -488,7 +502,7 @@ Use repeatable `--skip-target=<id-or-name>` or `--skip-target <id-or-name>` to e
|
|
|
488
502
|
|
|
489
503
|
**Output**: Console table summary + compact text summary for CI integrations.
|
|
490
504
|
|
|
491
|
-
**CI**: [.github/workflows/parser-health-check.yml](./.github/workflows/parser-health-check.yml) runs daily at 08:00 UTC (17:00 KST), or manually, on an `org-linux` runner with a 30-minute job timeout. It skips the two KHS excavation report/site-open targets. For crawling targets, the health-check composes the same fetch as production — robots.txt gate, then the KRAS detail adapter — so disallowed boards are reported as skipped rather than failed:
|
|
505
|
+
**CI**: [.github/workflows/parser-health-check.yml](./.github/workflows/parser-health-check.yml) runs daily at 08:00 UTC (17:00 KST), or manually, on an `org-linux` runner with a 30-minute job timeout. It skips the two KHS excavation report/site-open targets. For crawling targets, the health-check composes the same fetch as production — robots.txt gate, then the KRAS detail adapter — so disallowed boards are reported as skipped rather than failed: 17 skipped (including the injected journal issue target, which has no live board) and 53 checked with the current configuration. The detail check tries up to three list items, so one unreadable post at the top of a board does not fail the target. Slack notifications require the `SLACK_BOT_TOKEN` secret and `SLACK_ALERT_DEV_CHANNEL` repository variable. Forks need a matching runner and notification configuration to use this workflow unchanged. The CLI also writes GitHub Actions outputs and a job summary when their environment variables are present.
|
|
492
506
|
|
|
493
507
|
## 🤝 Contributing
|
|
494
508
|
|
package/dist/index.d.ts
CHANGED
|
@@ -7,8 +7,24 @@ interface VerifiedDeadlineReminder {
|
|
|
7
7
|
deadline: ArticleDeadline;
|
|
8
8
|
url: UrlString;
|
|
9
9
|
verificationStatus: 'verified';
|
|
10
|
+
/**
|
|
11
|
+
* Published issues in the past week that already listed this action.
|
|
12
|
+
* Host-supplied history; it only informs the reminder selection prompt.
|
|
13
|
+
*/
|
|
14
|
+
recentReminderCount?: number;
|
|
15
|
+
}
|
|
16
|
+
/** Days before the date an action first becomes eligible (D-3 through D-day). */
|
|
17
|
+
declare const DEADLINE_REMINDER_LEAD_DAYS = 3;
|
|
18
|
+
/** Rows the unified deadline section displays. */
|
|
19
|
+
declare const DEADLINE_REMINDER_LIMIT = 10;
|
|
20
|
+
/** A reminder that passed every display guard, with its normalized link. */
|
|
21
|
+
interface EligibleDeadlineReminder {
|
|
22
|
+
reminder: VerifiedDeadlineReminder;
|
|
23
|
+
/** Calendar days from publication to the start date; negative while ongoing. */
|
|
24
|
+
days: number;
|
|
25
|
+
url: string;
|
|
10
26
|
}
|
|
11
|
-
/** Deterministic deadlines/events, no LLM context
|
|
27
|
+
/** Deterministic deadlines/events, no LLM context: the first rows by date. */
|
|
12
28
|
declare function renderVerifiedDeadlineReminders(reminders: VerifiedDeadlineReminder[], publicationDate: IsoDateString, now?: Date): string;
|
|
13
29
|
|
|
14
30
|
/**
|
|
@@ -110,6 +126,77 @@ interface ExcavationReport {
|
|
|
110
126
|
* Called at most once per generation run.
|
|
111
127
|
*/
|
|
112
128
|
type ExcavationReportSource = () => Promise<ExcavationReport[]>;
|
|
129
|
+
/**
|
|
130
|
+
* One paper in an injected journal issue.
|
|
131
|
+
*/
|
|
132
|
+
interface JournalIssuePaper {
|
|
133
|
+
/** Paper title. */
|
|
134
|
+
title: string;
|
|
135
|
+
/** Author names in publication order. */
|
|
136
|
+
authors: string[];
|
|
137
|
+
/** Author keywords. Blank and repeated keywords are ignored. */
|
|
138
|
+
keywords: string[];
|
|
139
|
+
/**
|
|
140
|
+
* Abstract shown to the scoring and generation models, in the original
|
|
141
|
+
* language when the source has one. Truncated before it reaches them.
|
|
142
|
+
*/
|
|
143
|
+
abstract?: string | null;
|
|
144
|
+
/** Public page for the paper, such as its KCI article page. */
|
|
145
|
+
url: string;
|
|
146
|
+
}
|
|
147
|
+
/**
|
|
148
|
+
* A newly published journal issue (권호) supplied by the application.
|
|
149
|
+
*
|
|
150
|
+
* Each issue becomes one article whose body lists every paper it carries, so a
|
|
151
|
+
* twenty-paper issue is scored and introduced once rather than twenty times.
|
|
152
|
+
*/
|
|
153
|
+
interface JournalIssue {
|
|
154
|
+
/**
|
|
155
|
+
* Stable identifier for the issue, such as
|
|
156
|
+
* `{journalId}|{year}|{month}|{volume}|{issue}`. It becomes the article's
|
|
157
|
+
* `uniqId` and reaches `saveCrawledArticles`, so the application can store
|
|
158
|
+
* it with the article. `usedArticles` carry only the article's own id and
|
|
159
|
+
* URL, so that stored value (or the issue `url`) is how the application maps
|
|
160
|
+
* a saved newsletter back to the issues it introduced. Keep it independent
|
|
161
|
+
* of the paper list, or a later snapshot of the same issue looks like a new
|
|
162
|
+
* one.
|
|
163
|
+
*/
|
|
164
|
+
externalId: string;
|
|
165
|
+
/**
|
|
166
|
+
* Public page readers follow from the newsletter. It is also the article's
|
|
167
|
+
* URL, so it must be stable for the issue: previously saved issues are
|
|
168
|
+
* recognized by it and not saved again.
|
|
169
|
+
*/
|
|
170
|
+
url: string;
|
|
171
|
+
/** Journal name, e.g. "헤리티지:역사와 과학". */
|
|
172
|
+
journalName: string;
|
|
173
|
+
/** Publication year. */
|
|
174
|
+
publishedYear: number;
|
|
175
|
+
/** Publication month, 1-12. */
|
|
176
|
+
publishedMonth: number;
|
|
177
|
+
/** Volume (권). Empty when the journal numbers issues only. */
|
|
178
|
+
volume: string;
|
|
179
|
+
/** Issue number (호). Empty when the journal numbers volumes only. */
|
|
180
|
+
issue: string;
|
|
181
|
+
/**
|
|
182
|
+
* Date the issue became available to the application, in ISO format
|
|
183
|
+
* (YYYY-MM-DD, Korean time) — for example when its papers were first
|
|
184
|
+
* collected. Used as the article's registered date.
|
|
185
|
+
* @example "2026-10-28"
|
|
186
|
+
*/
|
|
187
|
+
registeredDate: IsoDateString;
|
|
188
|
+
/** Every paper in the issue, in the order to list them. */
|
|
189
|
+
papers: JournalIssuePaper[];
|
|
190
|
+
}
|
|
191
|
+
/**
|
|
192
|
+
* Supplies newly published journal issues from the application, such as those
|
|
193
|
+
* collected from KCI.
|
|
194
|
+
*
|
|
195
|
+
* The journal issue target has no live board behind it: when this is omitted
|
|
196
|
+
* the target collects nothing and makes no request. Called at most once per
|
|
197
|
+
* generation run.
|
|
198
|
+
*/
|
|
199
|
+
type JournalIssueSource = () => Promise<JournalIssue[]>;
|
|
113
200
|
/**
|
|
114
201
|
* Repository interface for tag management
|
|
115
202
|
*/
|
|
@@ -346,6 +433,17 @@ interface NewsletterGeneratorDependencies {
|
|
|
346
433
|
* it to keep crawling the board as before. No other target is affected.
|
|
347
434
|
*/
|
|
348
435
|
excavationReportSource?: ExcavationReportSource;
|
|
436
|
+
/**
|
|
437
|
+
* Supplies newly published journal issues from the application (optional),
|
|
438
|
+
* such as those collected from KCI.
|
|
439
|
+
*
|
|
440
|
+
* Each issue becomes one article listing its papers. The target has no live
|
|
441
|
+
* board, so omitting this leaves it empty without making a request. The
|
|
442
|
+
* article's `uniqId` is the issue's `externalId` and its URL the issue's
|
|
443
|
+
* `url`; store either when articles are saved to record which issues a
|
|
444
|
+
* saved newsletter introduced.
|
|
445
|
+
*/
|
|
446
|
+
journalIssueSource?: JournalIssueSource;
|
|
349
447
|
/**
|
|
350
448
|
* data.go.kr service key for the 나라일터 and 알리오 job boards (optional).
|
|
351
449
|
*
|
|
@@ -386,6 +484,15 @@ interface NewsletterGeneratorDependencies {
|
|
|
386
484
|
*/
|
|
387
485
|
declare function generateNewsletter(dependencies: NewsletterGeneratorDependencies): Promise<string | number | null>;
|
|
388
486
|
|
|
487
|
+
/**
|
|
488
|
+
* Board URL of the journal issue target.
|
|
489
|
+
*
|
|
490
|
+
* There is no list page behind it: the adapter below answers it from the
|
|
491
|
+
* application's injected issues and never sends it. It points at KCI because
|
|
492
|
+
* that is where the issues come from, and it is what the source list shows.
|
|
493
|
+
*/
|
|
494
|
+
declare const JOURNAL_ISSUE_LIST_URL = "https://www.kci.go.kr/kciportal/main.kci";
|
|
495
|
+
|
|
389
496
|
/**
|
|
390
497
|
* Generates a welcome email HTML string with CSS inlined via juice.
|
|
391
498
|
*
|
|
@@ -512,6 +619,78 @@ declare class AnalysisProvider implements AnalysisProvider$1 {
|
|
|
512
619
|
update(article: ArticleForUpdateByAnalysis): Promise<void>;
|
|
513
620
|
}
|
|
514
621
|
|
|
622
|
+
/**
|
|
623
|
+
* Chooses which eligible reminders the unified section displays.
|
|
624
|
+
*
|
|
625
|
+
* Returns a subset of the given rows in their original display order.
|
|
626
|
+
*/
|
|
627
|
+
type DeadlineReminderSelector = (rows: EligibleDeadlineReminder[], publicationDate: IsoDateString) => Promise<EligibleDeadlineReminder[]>;
|
|
628
|
+
type DeadlineReminderSelectorOptions = {
|
|
629
|
+
/** Same model that scores article importance. */
|
|
630
|
+
model: LanguageModel;
|
|
631
|
+
/**
|
|
632
|
+
* Milliseconds before the selection gives up and falls back. It runs while
|
|
633
|
+
* saving a fully generated issue, so a stalled provider must not hold it.
|
|
634
|
+
* @default 60000
|
|
635
|
+
*/
|
|
636
|
+
timeoutMs?: number;
|
|
637
|
+
/** Reports a selection that fell back to the first rows by date. */
|
|
638
|
+
onFallback?: (reason: string, candidateCount: number) => void;
|
|
639
|
+
/** Reports a model selection and its token usage. */
|
|
640
|
+
onSelected?: (info: {
|
|
641
|
+
candidateCount: number;
|
|
642
|
+
selectedCount: number;
|
|
643
|
+
/** Rows added by date order because the model chose too few. */
|
|
644
|
+
filledCount: number;
|
|
645
|
+
usage: unknown;
|
|
646
|
+
}) => void;
|
|
647
|
+
};
|
|
648
|
+
/**
|
|
649
|
+
* Builds the model selection that replaces "first rows by date".
|
|
650
|
+
*
|
|
651
|
+
* Several sources often announce the same 학술대회, and a busy day can hold
|
|
652
|
+
* fifteen same-day bids, so the earliest rows are a poor proxy for what readers
|
|
653
|
+
* need. The model judges relevance and duplicates; everything that decides
|
|
654
|
+
* whether a row may appear at all stays in {@link eligibleDeadlineReminders}.
|
|
655
|
+
*
|
|
656
|
+
* It never fails the newsletter: an error, a timeout, an empty answer or
|
|
657
|
+
* unusable indices fall back to the first rows by date. A selection shorter
|
|
658
|
+
* than the rows available is filled in date order, skipping named duplicates.
|
|
659
|
+
*/
|
|
660
|
+
declare const createDeadlineReminderSelector: (options: DeadlineReminderSelectorOptions) => DeadlineReminderSelector;
|
|
661
|
+
|
|
662
|
+
/** Memoized access to one run's injected journal issues. */
|
|
663
|
+
type JournalIssueLoader = () => Promise<JournalIssue[]>;
|
|
664
|
+
|
|
665
|
+
/** Which papers of an issue the table shows, and which it summarizes. */
|
|
666
|
+
type JournalPaperSelection = {
|
|
667
|
+
/** Paper indices shown in the table, most interesting first. */
|
|
668
|
+
featured: number[];
|
|
669
|
+
/** Papers collapsed into a count and topic summary, in paper order. */
|
|
670
|
+
others: {
|
|
671
|
+
index: number;
|
|
672
|
+
topic: string | null;
|
|
673
|
+
}[];
|
|
674
|
+
};
|
|
675
|
+
/**
|
|
676
|
+
* Picks the papers the newsletter's readers would want to see, or returns null
|
|
677
|
+
* when no selection could be made — the section then lists every paper.
|
|
678
|
+
*/
|
|
679
|
+
type JournalPaperSelector = (issue: JournalIssue) => Promise<JournalPaperSelection | null>;
|
|
680
|
+
|
|
681
|
+
/**
|
|
682
|
+
* Injected journal issues are rendered by the provider rather than written by
|
|
683
|
+
* the model: each is a table of up to twenty titles and author lists, which
|
|
684
|
+
* the model would have to copy exactly.
|
|
685
|
+
*/
|
|
686
|
+
type JournalIssueRendering = {
|
|
687
|
+
/** The run's injected issues, matched to candidates by URL. */
|
|
688
|
+
load: JournalIssueLoader;
|
|
689
|
+
/** Picks the papers readers would care about; omitted, every paper is listed. */
|
|
690
|
+
select?: JournalPaperSelector;
|
|
691
|
+
/** Reports that the issues could not be loaded, so the model got them. */
|
|
692
|
+
onUnavailable?: (reason: string) => void;
|
|
693
|
+
};
|
|
515
694
|
/**
|
|
516
695
|
* Content generation provider implementation
|
|
517
696
|
* - LLM-based newsletter content generation
|
|
@@ -522,15 +701,21 @@ declare class ContentGenerateProvider implements ContentGenerateProvider$1 {
|
|
|
522
701
|
private readonly articleRepository;
|
|
523
702
|
private readonly newsletterRepository;
|
|
524
703
|
private readonly templateOptions?;
|
|
704
|
+
/** Chooses displayed reminders; omitted, the first rows by date display. */
|
|
705
|
+
private readonly selectDeadlineReminders?;
|
|
706
|
+
private readonly journalIssues?;
|
|
525
707
|
private _issueOrder;
|
|
526
708
|
private readonly deadlineRemindersMarkdown;
|
|
709
|
+
private heldJournalIssues;
|
|
527
710
|
readonly model: LanguageModel;
|
|
528
711
|
readonly fetchRecentTitles?: ContentGenerateProvider$1['fetchRecentTitles'];
|
|
529
712
|
/** HTML template with markers for title and content injection */
|
|
530
713
|
htmlTemplate: HtmlTemplate;
|
|
531
714
|
/** Newsletter brand name (defaults to config, can be overridden via constructor) */
|
|
532
715
|
newsletterBrandName: string;
|
|
533
|
-
constructor(model: LanguageModel, articleRepository: ArticleRepository, newsletterRepository: NewsletterRepository, templateOptions?: NewsletterTemplateOptions | undefined, brandName?: string
|
|
716
|
+
constructor(model: LanguageModel, articleRepository: ArticleRepository, newsletterRepository: NewsletterRepository, templateOptions?: NewsletterTemplateOptions | undefined, brandName?: string,
|
|
717
|
+
/** Chooses displayed reminders; omitted, the first rows by date display. */
|
|
718
|
+
selectDeadlineReminders?: DeadlineReminderSelector | undefined, journalIssues?: JournalIssueRendering | undefined);
|
|
534
719
|
/** LLM temperature setting for content generation */
|
|
535
720
|
temperature: number;
|
|
536
721
|
/** Subscribe page URL */
|
|
@@ -565,6 +750,8 @@ declare class ContentGenerateProvider implements ContentGenerateProvider$1 {
|
|
|
565
750
|
}): Promise<{
|
|
566
751
|
id: string | number;
|
|
567
752
|
}>;
|
|
753
|
+
private renderJournalIssues;
|
|
754
|
+
private renderDeadlineReminders;
|
|
568
755
|
}
|
|
569
756
|
|
|
570
757
|
type HeritageBidCandidate = {
|
|
@@ -597,7 +784,7 @@ declare class CrawlingProvider implements CrawlingProvider$1 {
|
|
|
597
784
|
customFetch?: typeof fetch;
|
|
598
785
|
/** Crawling target groups configuration */
|
|
599
786
|
crawlingTargetGroups: CrawlingTargetGroup[];
|
|
600
|
-
constructor(articleRepository: ArticleRepository, customFetch?: typeof fetch, excavationReportSource?: ExcavationReportSource, logger?: AppLogger, publicDataApiKey?: string, bidTriage?: HeritageBidTriage);
|
|
787
|
+
constructor(articleRepository: ArticleRepository, customFetch?: typeof fetch, excavationReportSource?: ExcavationReportSource, logger?: AppLogger, publicDataApiKey?: string, bidTriage?: HeritageBidTriage, journalIssueSource?: JournalIssueSource);
|
|
601
788
|
/**
|
|
602
789
|
* Fetch existing articles by URLs to avoid duplicate crawling
|
|
603
790
|
* @param articleUrls - URLs to check
|
|
@@ -793,5 +980,5 @@ declare function fetchResearchRadarDeadlineSource({ source, customFetch, publicD
|
|
|
793
980
|
/** Only explicit, grounded results are verified. Missing dates are never cancellation. */
|
|
794
981
|
declare function refreshResearchRadarDeadlines(config: ResearchRadarDeadlineRefreshConfig): Promise<ResearchRadarDeadlineRefreshResult>;
|
|
795
982
|
|
|
796
|
-
export { AnalysisProvider, ContentGenerateProvider, CrawlingProvider, DateService, TaskService, contentOptions, createCrawlingTargetGroups, extractResearchRadarDeadlines, fetchResearchRadarDeadlineSource, generateNewsletter, generateWelcomeHTML, getSourceList, llmConfig, maximumImportanceScoreByDomain, newsletterConfig, refreshResearchRadarDeadlines, renderVerifiedDeadlineReminders, researchRadarPromptProvider, robotsExemptOrigins };
|
|
797
|
-
export type { ArticleRepository, ContentGenerationConfig, ContentOptions, DeadlineVerificationFailure, ExcavationReport, ExcavationReportSource, NewsletterConfig, NewsletterGeneratorDependencies, NewsletterModelConfig, NewsletterRepository, NewsletterTemplateOptions, PreviewNewsletterOptions, ResearchRadarDeadlineExtractionConfig, ResearchRadarDeadlineRefreshConfig, ResearchRadarDeadlineRefreshResult, ResearchRadarDeadlineSource, ResearchRadarDeadlineSourceOptions, ResearchRadarDeadlineSourceResult, SourceGroup, SourceItem, TagRepository, TaskRepository, VerifiedDeadlineReminder, WelcomeTemplateOptions };
|
|
983
|
+
export { AnalysisProvider, ContentGenerateProvider, CrawlingProvider, DEADLINE_REMINDER_LEAD_DAYS, DEADLINE_REMINDER_LIMIT, DateService, JOURNAL_ISSUE_LIST_URL, TaskService, contentOptions, createCrawlingTargetGroups, createDeadlineReminderSelector, extractResearchRadarDeadlines, fetchResearchRadarDeadlineSource, generateNewsletter, generateWelcomeHTML, getSourceList, llmConfig, maximumImportanceScoreByDomain, newsletterConfig, refreshResearchRadarDeadlines, renderVerifiedDeadlineReminders, researchRadarPromptProvider, robotsExemptOrigins };
|
|
984
|
+
export type { ArticleRepository, ContentGenerationConfig, ContentOptions, DeadlineReminderSelector, DeadlineReminderSelectorOptions, DeadlineVerificationFailure, EligibleDeadlineReminder, ExcavationReport, ExcavationReportSource, JournalIssue, JournalIssuePaper, JournalIssueSource, NewsletterConfig, NewsletterGeneratorDependencies, NewsletterModelConfig, NewsletterRepository, NewsletterTemplateOptions, PreviewNewsletterOptions, ResearchRadarDeadlineExtractionConfig, ResearchRadarDeadlineRefreshConfig, ResearchRadarDeadlineRefreshResult, ResearchRadarDeadlineSource, ResearchRadarDeadlineSourceOptions, ResearchRadarDeadlineSourceResult, SourceGroup, SourceItem, TagRepository, TaskRepository, VerifiedDeadlineReminder, WelcomeTemplateOptions };
|