@giacomo-ciro/paperino 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,65 @@
1
+ <h1 align="center">paperino</h1>
2
+
3
+ > Every new arXiv paper, every day, filtered by Claude Code down to what's actually worth your time.
4
+
5
+
6
+ <p align="center">
7
+ <img src="public/paperino.png" alt="Paperino banner" width="100%">
8
+ </p>
9
+
10
+ I work on a 3D vision project. On average, ~200 new papers are published daily in the cs.CV category.
11
+
12
+ Of those, ~40 look relevant from the title, and fewer than 5 are actually worth reading after scanning the abstract.
13
+
14
+ Existing tools based on similarity search or embeddings (Scholar Inbox, arXiv Sanity, etc.) aren't precise enough.
15
+
16
+ Claude Code delivers hyper-precise filtering tailored to your exact research project, running fresh every day.
17
+
18
+ Scanning the results then takes just 5-10 minutes, and you stay up to date with the latest research.
19
+
20
+ ## Usage
21
+ Paperino is a CLI utility, fully configurable via a simple .toml file. For now, it only produces an HTML digest; email delivery is coming soon.
22
+
23
+ You can run it manually:
24
+ ```
25
+ paperino
26
+ ```
27
+
28
+ Useful flags:
29
+ ```
30
+ paperino --configure # open the config file in your default editor
31
+ paperino --logs # tail the log file; no pipeline run
32
+ paperino --force # discard the run/digest for the selected window(s) and start fresh
33
+ paperino --only-fetch # only fetch papers, skip the scoring pipeline
34
+ paperino --quiet # suppress progress output; print only the digest path
35
+ paperino -y # skip the confirmation prompt and run immediately
36
+ ```
37
+
38
+ Or, as I do, run every weekday at 9:30 AM. Open the crontab:
39
+ ```bash
40
+ crontab -e
41
+ ```
42
+ and add:
43
+ ```bash
44
+ # minute 30, hour 9, Mon-Fri
45
+ 30 9 * * 1-5 paperino -y --quiet
46
+ ```
47
+ > **Note:** arXiv announces new submissions at 20:00 ET on Sun/Mon/Tue/Wed/Thu. Running at 9:30 AM CET (3:30 ET) ensures the run always lands after the prior evening's announcement, catching all five announcements without needing to run on weekends.
48
+
49
+ ## How It Works
50
+
51
+ **Preliminaries:** arXiv publishes new papers 5 times a week, on Sun, Mon, Tue, Wed and Thu at 20:00 CET. Each publication includes papers submitted during the preceding submission window (14:00 CET to 14:00 the following day), except weekend submissions, which are aggregated and published Monday night. Full details on the [official page](https://info.arxiv.org/help/availability.html#Announcement%20Schedule).
52
+
53
+ Paperino is minimal, built to work efficiently. It follows a 3-step process:
54
+
55
+ 1. **Fetching papers:** fetch all arXiv papers published in a given window (by default, the latest submission window relative to when the command is run). This step only filters by arXiv category (cs.LM, cs.CV, etc.).
56
+ 2. **Coarse filtering:** Claude receives your research context and a batch of titles per call (default 20), and outputs a binary relevant/not-relevant judgment. This step is kept coarse: Claude defaults to marking papers as potentially relevant when unsure.
57
+ 3. **Fine filtering:** Claude receives a smaller batch of title+abstract pairs per call and scores each paper on a scale of 1-10. Papers scoring above 6 get a full summary in the HTML digest; the rest get a one-line mention.
58
+
59
+ All aforementioned parameters are configurable (what model to use, papers per call, max papers, threshold score etc.):
60
+ ```
61
+ paperino --configure
62
+ ```
63
+
64
+ ## Acknowledgments
65
+ This tool was initially inspired by [AlessandroMorosini/arxiv-digest](https://github.com/AlessandroMorosini/arxiv-digest). Code-wise, I took inspiration from [kunchenguid/gnhf](https://github.com/kunchenguid/gnhf).
@@ -0,0 +1,95 @@
1
+ # Config file for paperino.
2
+ # Required fields: RESEARCH.ARXIV_CAT and RESEARCH.RESEARCH_INTERESTS.
3
+ # Everything else has a sensible default — edit only if you want to.
4
+
5
+ # What to fetch, and what counts as relevant.
6
+ [RESEARCH]
7
+ # arXiv categories to fetch. Required: add at least one.
8
+ # Browse the full taxonomy at https://arxiv.org/category_taxonomy
9
+ # Examples:
10
+ # "cs.LG", # machine learning
11
+ # "cs.CV", # computer vision
12
+ # "cs.CL", # computation and language
13
+ # "cs.AI", # artificial intelligence
14
+ # "cs.RO", # robotics
15
+ # "stat.ML", # statistical ml
16
+ # must be in quote: e.g., ARXIV_CAT=["cs.CV"]
17
+ ARXIV_CAT = []
18
+ # Describe your research so claude can judge relevance. Required.
19
+ # Templated into both scoring prompts as {research_interests} — the more
20
+ # specific, the better the filtering.
21
+ # Example: "Incremental 3D object reconstruction from streamed point clouds
22
+ # using transformer-based geometric primitive decoders."
23
+ RESEARCH_INTERESTS = """
24
+ """
25
+ # Papers scoring below this (on the 1-10 scale below) get only a one-line
26
+ # mention in the digest instead of a full summary.
27
+ MIN_SCORE = 6
28
+
29
+ # Where results land.
30
+ [OUTPUT]
31
+ # Each run writes to <out-dir>/YYYY-MM-DD/{papers.json, digest.html}, where
32
+ # YYYY-MM-DD is the submission window's cutoff date (UTC), not the run date.
33
+ OUT_DIR = "~/.paperino/outputs"
34
+ # Every run appends here (the file is never rotated). `paperino --logs` tails it.
35
+ LOG_FILE = "~/.paperino/paperino.log"
36
+
37
+ # Coarse pass: a cheap, fast model screens every fetched paper by title alone,
38
+ # marking it 0 or 1 to decide whether it's worth the pricier fine pass below.
39
+ # On error or malformed output, papers fail open (kept, not dropped).
40
+ [STAGES.COARSE]
41
+ MODEL = "haiku" # one of: fable, opus, sonnet, haiku
42
+ CALL_SIZE = 20 # papers per claude call
43
+ MAX_WORKERS = 10 # concurrent calls
44
+ # Only describe the task here — the output shape (JSON keys/types) is
45
+ # enforced separately via claude's structured-output schema (src/schemas.ts).
46
+ # Don't add a JSON example; anything below is task instructions only.
47
+ PROMPT = """
48
+ You are screening newly announced arXiv papers for relevance to a research project.
49
+
50
+ Project description:
51
+ {research_interests}
52
+
53
+ Below is a list of papers, one per line, formatted as "<paperId> — <title>".
54
+ Judging from the title alone, mark each paper 1 if it could plausibly be
55
+ relevant to the project, or 0 if it is clearly not relevant. This is a coarse
56
+ first pass: when in doubt, keep the paper (mark it 1).
57
+
58
+ Papers:
59
+ {papers}
60
+ """
61
+
62
+ # Fine pass: a stronger model scores every paper that passed the coarse pass,
63
+ # reading title and abstract, on the 1-10 scale below. A failed call leaves
64
+ # its papers unscored rather than guessing.
65
+ [STAGES.FINE]
66
+ MODEL = "sonnet" # one of: fable, opus, sonnet, haiku
67
+ CALL_SIZE = 5 # papers per claude call
68
+ MAX_WORKERS = 10 # concurrent calls
69
+ PROMPT = """
70
+ You are scoring newly announced arXiv papers for relevance to a research project.
71
+
72
+ Project description:
73
+ {research_interests}
74
+
75
+ Score each paper below on this scale:
76
+ - 9-10: Directly addresses the project's core problem or uses the same methods. Must-read.
77
+ - 7-8: Closely related — same domain, complementary techniques, useful as baseline.
78
+ - 5-6: Tangentially related — may contain reusable ideas or methods.
79
+ - 3-4: Loosely connected — different problem but same general area.
80
+ - 1-2: Not relevant to this specific project.
81
+
82
+ Papers:
83
+ {papers}
84
+ """
85
+
86
+ # How paperino calls claude.
87
+ [RUNTIME]
88
+ # Absolute path to the claude CLI binary, used for every scoring call.
89
+ # Auto-filled from `which claude` on first run. If that failed, run
90
+ # `which claude` yourself and paste the result here.
91
+ CLAUDE_BINARY = "claude"
92
+ # Per-call timeout, in seconds.
93
+ CALL_TIMEOUT_SECONDS = 300
94
+ # Retries after a call fails, before giving up on it.
95
+ CALL_RETRIES = 1
package/dist/cli.mjs ADDED
@@ -0,0 +1,1179 @@
1
+ #!/usr/bin/env node
2
+ import { appendFileSync, closeSync, existsSync, mkdirSync, openSync, readFileSync, rmSync, writeFileSync } from "node:fs";
3
+ import { Command, InvalidArgumentError } from "commander";
4
+ import pc from "picocolors";
5
+ import { execFileSync, spawn, spawnSync } from "node:child_process";
6
+ import { homedir, tmpdir } from "node:os";
7
+ import { dirname, join } from "node:path";
8
+ import { parse } from "smol-toml";
9
+ import { XMLParser } from "fast-xml-parser";
10
+ import { createInterface } from "node:readline/promises";
11
+ import cliSpinners from "cli-spinners";
12
+ //#region src/agent/claude.ts
13
+ function isFinalStructuredResult(event) {
14
+ return !event.is_error && event.subtype === "success" && !!event.structured_output;
15
+ }
16
+ function buildClaudeArgs(prompt, model, schema) {
17
+ return [
18
+ "--model",
19
+ model,
20
+ "-p",
21
+ prompt,
22
+ "--verbose",
23
+ "--output-format",
24
+ "stream-json",
25
+ "--json-schema",
26
+ JSON.stringify(schema),
27
+ "--allowedTools",
28
+ "StructuredOutput",
29
+ "--dangerously-skip-permissions"
30
+ ];
31
+ }
32
+ function terminateClaudeProcess(child) {
33
+ if (child.pid) try {
34
+ process.kill(-child.pid, "SIGTERM");
35
+ return;
36
+ } catch {}
37
+ child.kill("SIGTERM");
38
+ }
39
+ /** Split a newline-delimited JSON stream into parsed events, tolerating partial chunks. */
40
+ function readJSONLStream(stream, onEvent) {
41
+ let buffer = "";
42
+ stream.on("data", (chunk) => {
43
+ buffer += chunk.toString();
44
+ const lines = buffer.split("\n");
45
+ buffer = lines.pop() ?? "";
46
+ for (const line of lines) {
47
+ const trimmed = line.trim();
48
+ if (!trimmed) continue;
49
+ try {
50
+ onEvent(JSON.parse(trimmed));
51
+ } catch {}
52
+ }
53
+ });
54
+ }
55
+ var ClaudeAgent = class {
56
+ bin;
57
+ name = "claude";
58
+ constructor(bin) {
59
+ this.bin = bin;
60
+ }
61
+ run(prompt, opts) {
62
+ return new Promise((resolve, reject) => {
63
+ const child = spawn(this.bin, buildClaudeArgs(prompt, opts.model, opts.schema), {
64
+ cwd: tmpdir(),
65
+ detached: true,
66
+ stdio: [
67
+ "ignore",
68
+ "pipe",
69
+ "pipe"
70
+ ],
71
+ env: process.env
72
+ });
73
+ let stderr = "";
74
+ let settled = false;
75
+ const onAbort = () => {
76
+ if (settled) return;
77
+ settled = true;
78
+ terminateClaudeProcess(child);
79
+ reject(/* @__PURE__ */ new Error("claude call aborted"));
80
+ };
81
+ opts.signal?.addEventListener("abort", onAbort, { once: true });
82
+ child.stderr.on("data", (data) => {
83
+ stderr += data.toString();
84
+ });
85
+ child.on("error", (err) => {
86
+ if (settled) return;
87
+ settled = true;
88
+ reject(/* @__PURE__ */ new Error(`Failed to spawn claude: ${err.message}`));
89
+ });
90
+ child.stdout.on("error", (err) => {
91
+ if (settled) return;
92
+ settled = true;
93
+ reject(/* @__PURE__ */ new Error(`stdout stream error: ${err.message}`));
94
+ });
95
+ child.stderr.on("error", (err) => {
96
+ if (settled) return;
97
+ settled = true;
98
+ reject(/* @__PURE__ */ new Error(`stderr stream error: ${err.message}`));
99
+ });
100
+ let resultEvent = null;
101
+ let finalStructuredResultEvent = null;
102
+ readJSONLStream(child.stdout, (event) => {
103
+ if (event.type !== "result") return;
104
+ const next = event;
105
+ if (isFinalStructuredResult(next)) finalStructuredResultEvent = next;
106
+ else if (!finalStructuredResultEvent) resultEvent = next;
107
+ });
108
+ child.on("close", (code) => {
109
+ if (settled) return;
110
+ settled = true;
111
+ opts.signal?.removeEventListener("abort", onAbort);
112
+ if (code !== 0) {
113
+ reject(/* @__PURE__ */ new Error(`claude exited with code ${code}: ${stderr}`));
114
+ return;
115
+ }
116
+ const terminalResultEvent = finalStructuredResultEvent ?? resultEvent;
117
+ if (!terminalResultEvent) {
118
+ reject(/* @__PURE__ */ new Error("claude returned no result event"));
119
+ return;
120
+ }
121
+ if (terminalResultEvent.is_error || terminalResultEvent.subtype !== "success") {
122
+ reject(/* @__PURE__ */ new Error(`claude reported error: ${JSON.stringify(terminalResultEvent)}`));
123
+ return;
124
+ }
125
+ if (!terminalResultEvent.structured_output) {
126
+ reject(/* @__PURE__ */ new Error("claude returned no structured_output"));
127
+ return;
128
+ }
129
+ resolve({ output: terminalResultEvent.structured_output });
130
+ });
131
+ });
132
+ }
133
+ };
134
+ const CONFIG_PATH = join(join(homedir(), ".paperino"), "config.toml");
135
+ const ALLOWED_MODELS = [
136
+ "fable",
137
+ "opus",
138
+ "sonnet",
139
+ "haiku"
140
+ ];
141
+ const ARXIV_CATEGORY_RE = /^[a-z-]+(\.[A-Z]{2,3})?$/;
142
+ function expandTilde(path) {
143
+ return path.startsWith("~") ? join(homedir(), path.slice(1)) : path;
144
+ }
145
+ function templatePath() {
146
+ return new URL("./bootstrap-config.toml", import.meta.url).pathname;
147
+ }
148
+ function detectClaudeBinary() {
149
+ try {
150
+ return execFileSync("which", ["claude"], { encoding: "utf-8" }).trim() || null;
151
+ } catch {
152
+ return null;
153
+ }
154
+ }
155
+ /** `mkdir -p` the config dir and, on first run only, template the config from the bundled default. */
156
+ function ensureConfig(configPath = CONFIG_PATH) {
157
+ mkdirSync(dirname(configPath), { recursive: true });
158
+ if (existsSync(configPath)) return;
159
+ const template = readFileSync(templatePath(), "utf-8");
160
+ const claudeBinary = detectClaudeBinary();
161
+ const rendered = claudeBinary ? template.replace("CLAUDE_BINARY = \"claude\"", `CLAUDE_BINARY = "${claudeBinary}"`) : template;
162
+ if (!claudeBinary) process.stderr.write("warning: claude CLI not found — set RUNTIME.CLAUDE_BINARY in the config once it's installed.\n");
163
+ writeFileSync(configPath, rendered, "utf-8");
164
+ }
165
+ var ConfigErrorCollector = class {
166
+ errors = [];
167
+ add(message) {
168
+ this.errors.push(message);
169
+ }
170
+ string(table, key, label = key) {
171
+ const value = table[key];
172
+ if (typeof value !== "string") {
173
+ this.add(`"${label}" must be a string`);
174
+ return "";
175
+ }
176
+ return value;
177
+ }
178
+ nonEmptyString(table, key, label = key) {
179
+ const value = this.string(table, key, label);
180
+ if (value.trim() === "") this.add(`"${label}" is empty`);
181
+ return value;
182
+ }
183
+ number(table, key, label = key) {
184
+ const value = table[key];
185
+ if (typeof value !== "number") {
186
+ this.add(`"${label}" must be a number`);
187
+ return 0;
188
+ }
189
+ return value;
190
+ }
191
+ stringArray(table, key, label = key) {
192
+ const value = table[key];
193
+ if (!Array.isArray(value) || value.some((v) => typeof v !== "string")) {
194
+ this.add(`"${label}" must be an array of strings`);
195
+ return [];
196
+ }
197
+ return value;
198
+ }
199
+ nonEmptyStringArray(table, key, label = key) {
200
+ const value = this.stringArray(table, key, label);
201
+ if (value.length === 0) this.add(`"${label}" is empty`);
202
+ return value;
203
+ }
204
+ /** Number already extracted via `.number()`; checks it satisfies `predicate`, else records `detail`. */
205
+ numberSatisfies(value, label, predicate, detail) {
206
+ if (!predicate(value)) this.add(`"${label}" ${detail}`);
207
+ }
208
+ /** String already extracted via `.nonEmptyString()`; checks it's one of `allowed`. */
209
+ oneOf(value, allowed, label) {
210
+ if (value !== "" && !allowed.includes(value)) this.add(`"${label}" must be one of: ${allowed.join(", ")} (got "${value}")`);
211
+ }
212
+ /** Every arXiv category must look like "cs.LG" or "stat" (see https://arxiv.org/category_taxonomy). */
213
+ arxivCategories(value, label) {
214
+ for (const cat of value) if (!ARXIV_CATEGORY_RE.test(cat)) this.add(`"${label}" contains "${cat}", which doesn't look like an arXiv category (e.g. "cs.LG")`);
215
+ }
216
+ /** Prompt already extracted via `.nonEmptyString()`; must template both placeholders it's filled with. */
217
+ promptHasPlaceholders(value, label) {
218
+ if (value === "") return;
219
+ for (const placeholder of ["{research_interests}", "{papers}"]) if (!value.includes(placeholder)) this.add(`"${label}" is missing the ${placeholder} placeholder`);
220
+ }
221
+ table(table, key, label = key) {
222
+ const value = table[key];
223
+ if (typeof value !== "object" || value === null) {
224
+ this.add(`"[${label}]" table is missing`);
225
+ return {};
226
+ }
227
+ return value;
228
+ }
229
+ stage(stages, key, label) {
230
+ const stage = this.table(stages, key, label);
231
+ const model = this.nonEmptyString(stage, "MODEL", `${label}.MODEL`);
232
+ this.oneOf(model, ALLOWED_MODELS, `${label}.MODEL`);
233
+ const callSize = this.number(stage, "CALL_SIZE", `${label}.CALL_SIZE`);
234
+ this.numberSatisfies(callSize, `${label}.CALL_SIZE`, (n) => n > 0, "must be a positive number");
235
+ const maxWorkers = this.number(stage, "MAX_WORKERS", `${label}.MAX_WORKERS`);
236
+ this.numberSatisfies(maxWorkers, `${label}.MAX_WORKERS`, (n) => n > 0, "must be a positive number");
237
+ const prompt = this.nonEmptyString(stage, "PROMPT", `${label}.PROMPT`);
238
+ this.promptHasPlaceholders(prompt, `${label}.PROMPT`);
239
+ return {
240
+ model,
241
+ callSize,
242
+ maxWorkers,
243
+ prompt
244
+ };
245
+ }
246
+ throwIfAny(configPath) {
247
+ if (this.errors.length === 0) return;
248
+ const list = this.errors.map((e) => ` - ${e}`).join("\n");
249
+ throw new Error(`Config error(s) in ${configPath} (run --configure to edit):\n${list}`);
250
+ }
251
+ };
252
+ /** Load, validate, and map the UPPER_SNAKE_CASE config TOML. */
253
+ function loadConfig(configPath = CONFIG_PATH) {
254
+ const table = parse(readFileSync(configPath, "utf-8"));
255
+ const errors = new ConfigErrorCollector();
256
+ const runtime = errors.table(table, "RUNTIME");
257
+ const research = errors.table(table, "RESEARCH");
258
+ const output = errors.table(table, "OUTPUT");
259
+ const stages = errors.table(table, "STAGES");
260
+ const callTimeoutSeconds = errors.number(runtime, "CALL_TIMEOUT_SECONDS", "RUNTIME.CALL_TIMEOUT_SECONDS");
261
+ errors.numberSatisfies(callTimeoutSeconds, "RUNTIME.CALL_TIMEOUT_SECONDS", (n) => n > 0, "must be a positive number");
262
+ const callRetries = errors.number(runtime, "CALL_RETRIES", "RUNTIME.CALL_RETRIES");
263
+ errors.numberSatisfies(callRetries, "RUNTIME.CALL_RETRIES", (n) => n >= 0, "must be zero or a positive number");
264
+ const arxivCat = errors.nonEmptyStringArray(research, "ARXIV_CAT", "RESEARCH.ARXIV_CAT");
265
+ errors.arxivCategories(arxivCat, "RESEARCH.ARXIV_CAT");
266
+ const minScore = errors.number(research, "MIN_SCORE", "RESEARCH.MIN_SCORE");
267
+ errors.numberSatisfies(minScore, "RESEARCH.MIN_SCORE", (n) => n >= 1 && n <= 10, "must be between 1 and 10");
268
+ const config = {
269
+ claudeBinary: errors.nonEmptyString(runtime, "CLAUDE_BINARY", "RUNTIME.CLAUDE_BINARY"),
270
+ callTimeoutMs: callTimeoutSeconds * 1e3,
271
+ callRetries,
272
+ arxivCat,
273
+ researchInterests: errors.nonEmptyString(research, "RESEARCH_INTERESTS", "RESEARCH.RESEARCH_INTERESTS"),
274
+ minScore,
275
+ outDir: expandTilde(errors.nonEmptyString(output, "OUT_DIR", "OUTPUT.OUT_DIR")),
276
+ logFile: expandTilde(errors.nonEmptyString(output, "LOG_FILE", "OUTPUT.LOG_FILE")),
277
+ coarse: errors.stage(stages, "COARSE", "STAGES.COARSE"),
278
+ fine: errors.stage(stages, "FINE", "STAGES.FINE")
279
+ };
280
+ errors.throwIfAny(configPath);
281
+ return config;
282
+ }
283
+ /** Open the config file in the user's editor, falling back to `vi`. */
284
+ function openConfigInEditor(configPath = CONFIG_PATH) {
285
+ const candidates = [
286
+ process.env.VISUAL,
287
+ process.env.EDITOR,
288
+ "vi"
289
+ ].filter((c) => !!c);
290
+ for (const editor of candidates) {
291
+ const result = spawnSync(editor, [configPath], { stdio: "inherit" });
292
+ if (!result.error && result.status === 0) return;
293
+ }
294
+ process.stdout.write(`Could not open an editor. Edit the config file yourself: ${configPath}\n`);
295
+ }
296
+ //#endregion
297
+ //#region src/digest.ts
298
+ function escapeHtml(text) {
299
+ return text.replaceAll("&", "&amp;").replaceAll("<", "&lt;").replaceAll(">", "&gt;").replaceAll("\"", "&quot;").replaceAll("'", "&#39;");
300
+ }
301
+ /**
302
+ * Build (subject, html body) from all papers that passed the coarse filter.
303
+ * Papers scoring >= minScore get a full card; the rest (including papers left
304
+ * unscored by a failed call) get a one-line mention at the bottom.
305
+ */
306
+ function buildDigest(papers, windowDate, minScore) {
307
+ const kept = papers.filter((p) => p.coarse === 1).sort((a, b) => (b.score ?? 0) - (a.score ?? 0));
308
+ const top = kept.filter((p) => (p.score ?? 0) >= minScore);
309
+ const rest = kept.filter((p) => (p.score ?? 0) < minScore);
310
+ const subject = `arXiv digest — ${windowDate} (${top.length} papers)`;
311
+ if (kept.length === 0) return {
312
+ subject,
313
+ body: wrap(`<p><b>paperino:</b> submission window ${windowDate} — no relevant papers.</p>`)
314
+ };
315
+ const parts = [`<p><b>paperino:</b> submission window ${windowDate} — ${kept.length}/${papers.length} papers passed the coarse filter, ${top.length} scored ≥ ${minScore}.</p>`];
316
+ for (const p of top) parts.push(card(p));
317
+ if (rest.length > 0) {
318
+ const items = rest.map((p) => lineItem(p)).join("");
319
+ parts.push(`<h3>Lower-scored papers</h3><ul>${items}</ul>`);
320
+ }
321
+ return {
322
+ subject,
323
+ body: wrap(parts.join("\n"))
324
+ };
325
+ }
326
+ const HEART_SVG = `<svg width="12" height="12" viewBox="0 0 512 512" fill="currentColor" style="vertical-align:-1px" aria-hidden="true"><path d="M462.3 62.6C407.5 15.9 326 24.3 275.7 76.2L256 96.5l-19.7-20.3C186.1 24.3 104.5 15.9 49.7 62.6c-62.8 53.6-66.1 149.8-9.9 207.9l193.5 199.8c12.5 12.9 32.8 12.9 45.3 0l193.5-199.8c56.3-58.1 53-154.3-9.8-207.9z"/></svg>`;
327
+ function wrap(inner) {
328
+ return `<div style="max-width:900px;margin:0 auto;font-family:'Lucida Grande','Helvetica Neue',Helvetica,Arial,sans-serif;">
329
+
330
+ ${inner}
331
+
332
+ <hr style="margin-top:32px;border:none;border-top:1px solid #ddd;">
333
+ <p style="color:#888;font-size:90%;text-align:center"><a href="https://github.com/giacomo-ciro/paperino" style="color:inherit" target="_blank">paperino</a> - built by <a href="https://giacomociro.com" style="color:inherit" target="_blank">giacomo-ciro</a>, with ${HEART_SVG}</p>
334
+ </div>`;
335
+ }
336
+ function card(p) {
337
+ return `
338
+ <div style="margin-bottom:28px">
339
+ <h3 style="margin-bottom:4px"><a href="${p.link}">${escapeHtml(p.title)}</a></h3>
340
+ <p><b>Score: ${p.score}/10</b> — ${escapeHtml(p.summary ?? "")}</p>
341
+ <p><b>Key contribution:</b> ${escapeHtml(p.keyContribution ?? "")}</p>
342
+ <p><b>Why it matters:</b> ${escapeHtml(p.whyItMatters ?? "")}</p>
343
+ <p><b>Journal/Conference:</b> ${escapeHtml(p.journalRef ?? "n/a")}</p>
344
+ <p><b>Comment:</b> ${escapeHtml(p.comment ?? "n/a")}</p>
345
+ <p style="color:#555;font-size:90%">${escapeHtml(p.abstract ?? "")}</p>
346
+ </div>`;
347
+ }
348
+ function lineItem(p) {
349
+ const tag = p.score !== void 0 ? `score ${p.score}` : "unscored";
350
+ return `<li><a href="${p.link}">${escapeHtml(p.title)}</a> (${tag})</li>`;
351
+ }
352
+ //#endregion
353
+ //#region src/fetch.ts
354
+ const ARXIV_API_BASE = "https://export.arxiv.org/api/query";
355
+ const PAGE_SIZE = 100;
356
+ const INTER_PAGE_DELAY_MS = 3e3;
357
+ const parser = new XMLParser({
358
+ ignoreAttributes: false,
359
+ attributeNamePrefix: "@_",
360
+ isArray: (name) => [
361
+ "entry",
362
+ "category",
363
+ "author",
364
+ "link"
365
+ ].includes(name),
366
+ removeNSPrefix: true
367
+ });
368
+ function toPaper(entry) {
369
+ const id = entry.id.replace(/^https?:\/\/arxiv\.org\/abs\//, "").replaceAll("/", "_");
370
+ const link = entry.link.find((l) => l["@_rel"] === "alternate")?.["@_href"] ?? entry.id;
371
+ return {
372
+ id,
373
+ title: entry.title.trim(),
374
+ abstract: entry.summary.trim(),
375
+ link,
376
+ categories: entry.category.map((c) => c["@_term"]),
377
+ published: entry.published,
378
+ journalRef: entry.journal_ref ?? null,
379
+ comment: entry.comment ?? null
380
+ };
381
+ }
382
+ function formatArxivDate(d) {
383
+ const pad = (n) => String(n).padStart(2, "0");
384
+ return `${d.getUTCFullYear()}${pad(d.getUTCMonth() + 1)}${pad(d.getUTCDate())}${pad(d.getUTCHours())}${pad(d.getUTCMinutes())}`;
385
+ }
386
+ function sleep(ms) {
387
+ return new Promise((resolve) => setTimeout(resolve, ms));
388
+ }
389
+ /**
390
+ * Fetch every arXiv paper in `categories` submitted within [start, end] UTC.
391
+ * If `maxPapers` is set, only the most recently submitted N are returned.
392
+ */
393
+ async function fetchRecentPapers(categories, start, end, maxPapers) {
394
+ const dateFilter = `submittedDate:[${formatArxivDate(start)} TO ${formatArxivDate(end)}]`;
395
+ const searchQuery = `(${categories.map((c) => `cat:${c}`).join(" OR ")}) AND ${dateFilter}`;
396
+ const papers = [];
397
+ let start_ = 0;
398
+ let totalResults = Infinity;
399
+ while (start_ < totalResults && (maxPapers === void 0 || papers.length < maxPapers)) {
400
+ if (start_ > 0) await sleep(INTER_PAGE_DELAY_MS);
401
+ const url = new URL(ARXIV_API_BASE);
402
+ url.searchParams.set("search_query", searchQuery);
403
+ url.searchParams.set("start", String(start_));
404
+ url.searchParams.set("max_results", String(PAGE_SIZE));
405
+ url.searchParams.set("sortBy", "submittedDate");
406
+ url.searchParams.set("sortOrder", "descending");
407
+ const response = await fetch(url);
408
+ if (!response.ok) throw new Error(`arXiv API request failed: ${response.status} ${response.statusText}`);
409
+ const xml = await response.text();
410
+ const parsed = parser.parse(xml);
411
+ totalResults = parsed.feed.totalResults;
412
+ const entries = parsed.feed.entry ?? [];
413
+ if (entries.length === 0) break;
414
+ for (const entry of entries) {
415
+ papers.push(toPaper(entry));
416
+ if (maxPapers !== void 0 && papers.length >= maxPapers) break;
417
+ }
418
+ start_ += entries.length;
419
+ }
420
+ return maxPapers !== void 0 ? papers.slice(0, maxPapers) : papers;
421
+ }
422
+ //#endregion
423
+ //#region src/logger.ts
424
+ function timestamp() {
425
+ const now = /* @__PURE__ */ new Date();
426
+ const pad = (n) => String(n).padStart(2, "0");
427
+ return `${`${now.getFullYear()}-${pad(now.getMonth() + 1)}-${pad(now.getDate())}`} ${`${pad(now.getHours())}:${pad(now.getMinutes())}:${pad(now.getSeconds())}`}`;
428
+ }
429
+ function makeLogger(logPath) {
430
+ mkdirSync(dirname(logPath), { recursive: true });
431
+ closeSync(openSync(logPath, "a"));
432
+ function write(line) {
433
+ appendFileSync(logPath, `[${timestamp()}] ${line}\n`, "utf-8");
434
+ }
435
+ return {
436
+ pipelineStart(windowCount) {
437
+ write(`pipeline started (${windowCount} window(s) to process)`);
438
+ },
439
+ windowStart(label) {
440
+ write(`processing window ${label}`);
441
+ },
442
+ stageStart(label) {
443
+ write(` stage started: ${label}`);
444
+ },
445
+ callFailed(stageLabel, error) {
446
+ write(` call failed: ${stageLabel} — ${error}`);
447
+ },
448
+ stageEnd(label, metrics) {
449
+ write(` stage finished: ${label} — ${metrics}`);
450
+ },
451
+ pipelineEnd() {
452
+ write("pipeline finished");
453
+ }
454
+ };
455
+ }
456
+ /** Opens the log file in `less`, jumping to the end, inheriting stdio until the user quits. */
457
+ function viewLogs(logPath) {
458
+ mkdirSync(dirname(logPath), { recursive: true });
459
+ closeSync(openSync(logPath, "a"));
460
+ spawnSync("less", ["+G", logPath], { stdio: "inherit" });
461
+ }
462
+ //#endregion
463
+ //#region src/progress.ts
464
+ const STAGE_LABELS = [
465
+ "Fetching papers",
466
+ "Coarse filtering",
467
+ "Fine filtering"
468
+ ];
469
+ const SPINNER = cliSpinners.dots;
470
+ const ANSI_PATTERN = /\x1b\[[0-9;]*m/g;
471
+ /** Visible width of a line once SGR color codes are stripped. */
472
+ function visibleWidth(line) {
473
+ return line.replace(ANSI_PATTERN, "").length;
474
+ }
475
+ /** How many physical terminal rows a line occupies once wrapped at `columns`. */
476
+ function wrappedRows(line, columns) {
477
+ return Math.max(1, Math.ceil(visibleWidth(line) / columns));
478
+ }
479
+ /**
480
+ * Redraws a block of lines in place on stderr, tracking how many physical
481
+ * rows the last frame occupied (accounting for wrapping) so it can be
482
+ * cleared cleanly before the next frame — or erased entirely via `clear()`.
483
+ */
484
+ var Frame = class {
485
+ linesPrinted = 0;
486
+ draw(lines) {
487
+ const columns = process.stderr.columns || 80;
488
+ const totalRows = lines.map((line) => wrappedRows(line, columns)).reduce((sum, n) => sum + n, 0);
489
+ if (this.linesPrinted > 0) {
490
+ process.stderr.write(`\x1b[${this.linesPrinted}A`);
491
+ for (let i = 0; i < this.linesPrinted; i++) process.stderr.write(i === this.linesPrinted - 1 ? "\x1B[2K" : "\x1B[2K\x1B[1B");
492
+ if (this.linesPrinted > 1) process.stderr.write(`\x1b[${this.linesPrinted - 1}A`);
493
+ }
494
+ for (const line of lines) process.stderr.write(`${line}\n`);
495
+ this.linesPrinted = totalRows;
496
+ }
497
+ /**
498
+ * Erase everything this frame printed, leaving the cursor where the frame started.
499
+ * `extraLines` accounts for rows printed after the last `draw()` outside of Frame's
500
+ * control (e.g. the newline a TTY echoes when the user presses Enter at a prompt).
501
+ */
502
+ clear(extraLines = 0) {
503
+ const rows = this.linesPrinted + extraLines;
504
+ if (rows === 0) return;
505
+ process.stderr.write(`\x1b[${rows}A`);
506
+ for (let i = 0; i < rows; i++) process.stderr.write(i === rows - 1 ? "\x1B[2K" : "\x1B[2K\x1B[1B");
507
+ if (rows > 1) process.stderr.write(`\x1b[${rows - 1}A`);
508
+ this.linesPrinted = 0;
509
+ }
510
+ };
511
+ const ET_TIME_ZONE$1 = "America/New_York";
512
+ const LOCAL_TIME_ZONE = Intl.DateTimeFormat().resolvedOptions().timeZone;
513
+ const weekdayFormatter = new Intl.DateTimeFormat("en-US", {
514
+ weekday: "short",
515
+ timeZone: ET_TIME_ZONE$1
516
+ });
517
+ const etTimeFormatter = new Intl.DateTimeFormat("en-US", {
518
+ timeZone: ET_TIME_ZONE$1,
519
+ year: "numeric",
520
+ month: "2-digit",
521
+ day: "2-digit",
522
+ hour: "2-digit",
523
+ minute: "2-digit",
524
+ hourCycle: "h23"
525
+ });
526
+ const localTimeFormatter = new Intl.DateTimeFormat("en-US", {
527
+ timeZone: LOCAL_TIME_ZONE,
528
+ hour: "2-digit",
529
+ minute: "2-digit",
530
+ hourCycle: "h23",
531
+ timeZoneName: "short"
532
+ });
533
+ /** `YYYY-MM-DD (Ddd)` in ET, for the confirmation summary. */
534
+ function formatETDate(d) {
535
+ const parts = Object.fromEntries(etTimeFormatter.formatToParts(d).map((p) => [p.type, p.value]));
536
+ return `${`${parts.year}-${parts.month}-${parts.day}`} (${weekdayFormatter.format(d)})`;
537
+ }
538
+ /** `14:00 ET (your timezone: HH:mm <zone>)` — the cutoff time shared by both window boundaries. */
539
+ function formatCutoffTime(d) {
540
+ const parts = Object.fromEntries(etTimeFormatter.formatToParts(d).map((p) => [p.type, p.value]));
541
+ const etTime = `${parts.hour}:${parts.minute} ET`;
542
+ if (LOCAL_TIME_ZONE === ET_TIME_ZONE$1) return etTime;
543
+ const localParts = Object.fromEntries(localTimeFormatter.formatToParts(d).map((p) => [p.type, p.value]));
544
+ return `${etTime} (your timezone: ${`${localParts.hour}:${localParts.minute} ${localParts.timeZoneName}`})`;
545
+ }
546
+ /**
547
+ * Renders the run summary (windows, config, warnings) in the same visual
548
+ * language as the stage tracker, then prompts for confirmation. The whole
549
+ * block is wiped from the screen right before returning, so it doesn't
550
+ * linger once the animated stage tracker takes over.
551
+ */
552
+ async function confirmRun(windows, cfg, maxPapers, force, onlyFetch) {
553
+ const frame = new Frame();
554
+ const lines = [];
555
+ lines.push("");
556
+ lines.push(`${windows.length} submission window(s) to process${onlyFetch ? " (only fetching)" : ""}:`);
557
+ lines.push("");
558
+ for (const [start, end] of windows) lines.push(` ${pc.cyan("›")} ${formatETDate(start)} ${pc.dim("->")} ${formatETDate(end)}`);
559
+ if (windows.length > 0) {
560
+ const cutoff = formatCutoffTime(windows[windows.length - 1][1]);
561
+ lines.push("");
562
+ lines.push(pc.dim(`Each window starts and ends at ${cutoff}`));
563
+ }
564
+ lines.push("");
565
+ const maxProcessed = maxPapers !== void 0 ? String(maxPapers) : "all";
566
+ lines.push(`${pc.dim("Max papers/window")} ${maxProcessed}`);
567
+ if (!onlyFetch) {
568
+ lines.push(`${pc.dim("Coarse model")} ${cfg.coarse.model} (${cfg.coarse.callSize} papers per call, ${cfg.coarse.maxWorkers} concurrent calls at most)`);
569
+ lines.push(`${pc.dim("Fine model")} ${cfg.fine.model} (${cfg.fine.callSize} papers per call, ${cfg.fine.maxWorkers} concurrent calls at most)`);
570
+ }
571
+ if (force) {
572
+ lines.push("");
573
+ lines.push(pc.yellow("existing runs/digests for these windows will be discarded and rerun from scratch."));
574
+ }
575
+ lines.push("");
576
+ lines.push(pc.dim("Press Enter to proceed (any other key to abort)"));
577
+ frame.draw(lines);
578
+ const rl = createInterface({
579
+ input: process.stdin,
580
+ output: process.stderr
581
+ });
582
+ let confirmed;
583
+ try {
584
+ confirmed = (await rl.question("")).trim() === "";
585
+ } finally {
586
+ rl.close();
587
+ }
588
+ frame.clear(1);
589
+ return confirmed;
590
+ }
591
+ function makePipelineView(windowLabel, stageLabels = STAGE_LABELS) {
592
+ let index = 0;
593
+ let frameNum = 0;
594
+ let stopped = false;
595
+ const text = [...stageLabels];
596
+ const failedStages = /* @__PURE__ */ new Set();
597
+ const frame = new Frame();
598
+ let timer;
599
+ function renderLine(i) {
600
+ const step = pc.dim(`Step ${i + 1}`);
601
+ if (i < index) return `${step} ${failedStages.has(i) ? pc.yellow("✓") : pc.green("✓")} ${text[i]}`;
602
+ if (i === index && !stopped) return `${step} ${pc.cyan(SPINNER.frames[frameNum % SPINNER.frames.length])} ${text[i]}`;
603
+ return pc.dim(`Step ${i + 1} ○ ${text[i]}`);
604
+ }
605
+ function draw() {
606
+ frame.draw(stageLabels.map((_, i) => renderLine(i)));
607
+ }
608
+ process.stderr.write(`\nProcessing window ${windowLabel}\n\n`);
609
+ draw();
610
+ timer = setInterval(() => {
611
+ frameNum++;
612
+ draw();
613
+ }, SPINNER.interval);
614
+ return {
615
+ update(newText) {
616
+ text[index] = newText;
617
+ },
618
+ complete(finalText, failed = false) {
619
+ text[index] = finalText;
620
+ if (failed) failedStages.add(index);
621
+ index++;
622
+ if (index < stageLabels.length) text[index] = stageLabels[index];
623
+ },
624
+ stop() {
625
+ if (timer) clearInterval(timer);
626
+ stopped = true;
627
+ draw();
628
+ process.stderr.write("\n");
629
+ }
630
+ };
631
+ }
632
+ /** Non-interactive fallback (--quiet or non-TTY stderr): no animation, no output at all. */
633
+ function makeSilentPipelineView() {
634
+ return {
635
+ update() {},
636
+ complete() {},
637
+ stop() {}
638
+ };
639
+ }
640
+ //#endregion
641
+ //#region src/agent/pool.ts
642
+ async function runOnce(agent, prompt, opts) {
643
+ const controller = new AbortController();
644
+ const timer = setTimeout(() => controller.abort(), opts.timeoutMs);
645
+ try {
646
+ const { output } = await agent.run(prompt, {
647
+ model: opts.model,
648
+ schema: opts.schema,
649
+ timeoutMs: opts.timeoutMs,
650
+ signal: controller.signal
651
+ });
652
+ return output;
653
+ } finally {
654
+ clearTimeout(timer);
655
+ }
656
+ }
657
+ async function runWithRetries(agent, prompt, index, opts) {
658
+ const totalAttempts = opts.retries + 1;
659
+ let lastError;
660
+ for (let attempt = 1; attempt <= totalAttempts; attempt++) try {
661
+ return await runOnce(agent, prompt, opts);
662
+ } catch (err) {
663
+ lastError = err;
664
+ if (attempt < totalAttempts) opts.onAttemptFailed?.(err instanceof Error ? err.message : String(err), index, attempt, totalAttempts);
665
+ }
666
+ return lastError instanceof Error ? lastError : new Error(String(lastError));
667
+ }
668
+ /**
669
+ * Fan `prompts` out over a concurrency-limited pool. Returns one entry per prompt,
670
+ * in order: the parsed output, or an Error if that prompt ultimately failed after
671
+ * retries (so one bad call can't abort the run).
672
+ */
673
+ async function runPool(agent, prompts, opts) {
674
+ const results = new Array(prompts.length);
675
+ let next = 0;
676
+ let completed = 0;
677
+ async function worker() {
678
+ while (true) {
679
+ const index = next++;
680
+ if (index >= prompts.length) return;
681
+ const result = await runWithRetries(agent, prompts[index], index, opts);
682
+ results[index] = result;
683
+ completed++;
684
+ opts.onProgress?.(result, index, completed, prompts.length);
685
+ }
686
+ }
687
+ const workerCount = Math.min(opts.maxWorkers, prompts.length);
688
+ await Promise.all(Array.from({ length: workerCount }, () => worker()));
689
+ return results;
690
+ }
691
+ //#endregion
692
+ //#region src/schemas.ts
693
+ const COARSE_SCHEMA = {
694
+ type: "object",
695
+ properties: { verdicts: {
696
+ type: "array",
697
+ items: {
698
+ type: "object",
699
+ properties: {
700
+ id: { type: "string" },
701
+ coarse: {
702
+ type: "integer",
703
+ enum: [0, 1]
704
+ }
705
+ },
706
+ required: ["id", "coarse"],
707
+ additionalProperties: false
708
+ }
709
+ } },
710
+ required: ["verdicts"],
711
+ additionalProperties: false
712
+ };
713
+ const FINE_SCHEMA = {
714
+ type: "object",
715
+ properties: { papers: {
716
+ type: "array",
717
+ items: {
718
+ type: "object",
719
+ properties: {
720
+ id: { type: "string" },
721
+ score: {
722
+ type: "integer",
723
+ minimum: 1,
724
+ maximum: 10
725
+ },
726
+ summary: { type: "string" },
727
+ key_contribution: { type: "string" },
728
+ why_it_matters: { type: "string" }
729
+ },
730
+ required: [
731
+ "id",
732
+ "score",
733
+ "summary",
734
+ "key_contribution",
735
+ "why_it_matters"
736
+ ],
737
+ additionalProperties: false
738
+ }
739
+ } },
740
+ required: ["papers"],
741
+ additionalProperties: false
742
+ };
743
+ /**
744
+ * Replace only the two exact tokens `{research_interests}` and `{papers}` in `template`.
745
+ * Every other brace (e.g. the prompt's inline JSON examples) is left untouched.
746
+ */
747
+ function fillPrompt(template, values) {
748
+ return template.replaceAll("{research_interests}", values.research_interests).replaceAll("{papers}", values.papers);
749
+ }
750
+ //#endregion
751
+ //#region src/scoring.ts
752
+ /** Group items into chunks of `size`, one chunk per claude call. */
753
+ function groupForCalls(items, size) {
754
+ const groups = [];
755
+ for (let i = 0; i < items.length; i += size) groups.push(items.slice(i, i + size));
756
+ return groups;
757
+ }
758
+ function coarseFlag(verdicts, paperId) {
759
+ const raw = verdicts.get(paperId);
760
+ const parsed = typeof raw === "number" ? raw : Number(raw);
761
+ return Number.isFinite(parsed) && parsed === 0 ? 0 : 1;
762
+ }
763
+ /** Title-only screening: mark each paper coarse: 0|1. Fail-open on errors. */
764
+ async function coarseFilter(papers, cfg, agent, onProgress, onCallFailed) {
765
+ const pending = papers.filter((p) => p.coarse === void 0);
766
+ if (pending.length === 0) return;
767
+ const calls = groupForCalls(pending, cfg.coarse.callSize);
768
+ const prompts = calls.map((c) => fillPrompt(cfg.coarse.prompt, {
769
+ research_interests: cfg.researchInterests,
770
+ papers: c.map((p) => `${p.id} — ${p.title}`).join("\n")
771
+ }));
772
+ let papersDone = 0;
773
+ let kept = 0;
774
+ let failed = 0;
775
+ onProgress?.({
776
+ papersDone: 0,
777
+ papersTotal: pending.length,
778
+ passed: 0,
779
+ callsDone: 0,
780
+ callsTotal: calls.length,
781
+ failed: 0
782
+ });
783
+ await runPool(agent, prompts, {
784
+ model: cfg.coarse.model,
785
+ schema: COARSE_SCHEMA,
786
+ maxWorkers: cfg.coarse.maxWorkers,
787
+ timeoutMs: cfg.callTimeoutMs,
788
+ retries: cfg.callRetries,
789
+ onAttemptFailed: (error, _index, attempt, totalAttempts) => {
790
+ onCallFailed?.(`attempt ${attempt}/${totalAttempts} failed, retrying — ${error}`);
791
+ },
792
+ onProgress: (output, index, callsDone, callsTotal) => {
793
+ const c = calls[index];
794
+ const entries = output && typeof output === "object" && !Array.isArray(output) && !(output instanceof Error) ? output.verdicts : void 0;
795
+ const callFailed = output instanceof Error || !Array.isArray(entries);
796
+ if (callFailed) {
797
+ failed++;
798
+ onCallFailed?.(output instanceof Error ? output.message : "malformed output shape");
799
+ }
800
+ const verdicts = /* @__PURE__ */ new Map();
801
+ if (!callFailed) {
802
+ for (const entry of entries) if (entry && typeof entry === "object" && "id" in entry) verdicts.set(String(entry.id), entry.coarse);
803
+ }
804
+ for (const p of c) {
805
+ p.coarse = callFailed ? 1 : coarseFlag(verdicts, p.id);
806
+ if (p.coarse === 1) kept++;
807
+ }
808
+ papersDone += c.length;
809
+ onProgress?.({
810
+ papersDone,
811
+ papersTotal: pending.length,
812
+ passed: kept,
813
+ callsDone,
814
+ callsTotal,
815
+ failed
816
+ });
817
+ }
818
+ });
819
+ }
820
+ /** Score coarse-passed papers on title+abstract; failed calls stay unscored. */
821
+ async function fineScoring(papers, cfg, agent, onProgress, onCallFailed) {
822
+ const pending = papers.filter((p) => p.coarse === 1 && p.score === void 0);
823
+ if (pending.length === 0) return;
824
+ const calls = groupForCalls(pending, cfg.fine.callSize);
825
+ const prompts = calls.map((c) => fillPrompt(cfg.fine.prompt, {
826
+ research_interests: cfg.researchInterests,
827
+ papers: c.map((p) => `id: ${p.id}\ntitle: ${p.title}\nabstract: ${p.abstract}`).join("\n\n")
828
+ }));
829
+ let papersDone = 0;
830
+ let scored = 0;
831
+ let failed = 0;
832
+ onProgress?.({
833
+ papersDone: 0,
834
+ papersTotal: pending.length,
835
+ passed: 0,
836
+ callsDone: 0,
837
+ callsTotal: calls.length,
838
+ failed: 0
839
+ });
840
+ await runPool(agent, prompts, {
841
+ model: cfg.fine.model,
842
+ schema: FINE_SCHEMA,
843
+ maxWorkers: cfg.fine.maxWorkers,
844
+ timeoutMs: cfg.callTimeoutMs,
845
+ retries: cfg.callRetries,
846
+ onAttemptFailed: (error, _index, attempt, totalAttempts) => {
847
+ onCallFailed?.(`attempt ${attempt}/${totalAttempts} failed, retrying — ${error}`);
848
+ },
849
+ onProgress: (output, index, callsDone, callsTotal) => {
850
+ const c = calls[index];
851
+ const entries = output && typeof output === "object" && !Array.isArray(output) && !(output instanceof Error) ? output.papers : void 0;
852
+ if (!(output instanceof Error) && Array.isArray(entries)) {
853
+ const byId = /* @__PURE__ */ new Map();
854
+ for (const entry of entries) if (entry && typeof entry === "object" && "id" in entry) byId.set(String(entry.id), entry);
855
+ for (const p of c) {
856
+ const entry = byId.get(p.id);
857
+ if (!entry) continue;
858
+ const score = Number(entry.score);
859
+ if (!Number.isFinite(score)) continue;
860
+ p.score = score;
861
+ p.summary = String(entry.summary ?? "");
862
+ p.keyContribution = String(entry.key_contribution ?? "");
863
+ p.whyItMatters = String(entry.why_it_matters ?? "");
864
+ scored++;
865
+ }
866
+ } else {
867
+ failed++;
868
+ onCallFailed?.(output instanceof Error ? output.message : "malformed output shape");
869
+ }
870
+ papersDone += c.length;
871
+ onProgress?.({
872
+ papersDone,
873
+ papersTotal: pending.length,
874
+ passed: scored,
875
+ callsDone,
876
+ callsTotal,
877
+ failed
878
+ });
879
+ }
880
+ });
881
+ }
882
+ //#endregion
883
+ //#region src/store.ts
884
+ /** `YYYY-MM-DD` for a UTC instant (matches Python's `strftime("%Y-%m-%d")`). */
885
+ function formatUTCDate(d) {
886
+ const pad = (n) => String(n).padStart(2, "0");
887
+ return `${d.getUTCFullYear()}-${pad(d.getUTCMonth() + 1)}-${pad(d.getUTCDate())}`;
888
+ }
889
+ /** One directory per submission window, named by its end date (idempotent across re-runs). */
890
+ function runDir(outDir, windowEnd) {
891
+ const dir = join(outDir, formatUTCDate(windowEnd));
892
+ mkdirSync(dir, { recursive: true });
893
+ return dir;
894
+ }
895
+ /** Delete a window's run directory (papers.json, digest.html) so it reruns from scratch. */
896
+ function clearRunDir(outDir, windowEnd) {
897
+ rmSync(join(outDir, formatUTCDate(windowEnd)), {
898
+ recursive: true,
899
+ force: true
900
+ });
901
+ }
902
+ function papersPath(dir) {
903
+ return join(dir, "papers.json");
904
+ }
905
+ function loadPapers(dir) {
906
+ const path = papersPath(dir);
907
+ if (!existsSync(path)) return [];
908
+ return JSON.parse(readFileSync(path, "utf-8"));
909
+ }
910
+ function savePapers(dir, papers) {
911
+ writeFileSync(papersPath(dir), `${JSON.stringify(papers, null, 2)}\n`, "utf-8");
912
+ }
913
+ /**
914
+ * Merge freshly-fetched papers into the existing store, keyed by id.
915
+ * Existing records (which may already carry `coarse`/`score`) win on conflict;
916
+ * records no longer returned by fetch are kept (never deleted).
917
+ */
918
+ function mergePapers(existing, fetched) {
919
+ const byId = /* @__PURE__ */ new Map();
920
+ for (const p of existing) byId.set(p.id, p);
921
+ for (const p of fetched) if (!byId.has(p.id)) byId.set(p.id, p);
922
+ return [...byId.values()];
923
+ }
924
+ function writeDigest(dir, html) {
925
+ const path = join(dir, "digest.html");
926
+ writeFileSync(path, html, "utf-8");
927
+ return path;
928
+ }
929
+ //#endregion
930
+ //#region src/window.ts
931
+ const ET_TIME_ZONE = "America/New_York";
932
+ const CUTOFF_HOUR = 14;
933
+ const etFormatter = new Intl.DateTimeFormat("en-US", {
934
+ timeZone: ET_TIME_ZONE,
935
+ year: "numeric",
936
+ month: "2-digit",
937
+ day: "2-digit",
938
+ hour: "2-digit",
939
+ minute: "2-digit",
940
+ second: "2-digit",
941
+ hourCycle: "h23"
942
+ });
943
+ /** Convert a UTC instant to its ET wall-clock parts. */
944
+ function utcToEtWallClock(instant) {
945
+ const parts = etFormatter.formatToParts(instant);
946
+ const get = (type) => Number(parts.find((p) => p.type === type)?.value);
947
+ return {
948
+ year: get("year"),
949
+ month: get("month"),
950
+ day: get("day"),
951
+ hour: get("hour"),
952
+ minute: get("minute"),
953
+ second: get("second")
954
+ };
955
+ }
956
+ /**
957
+ * Convert ET wall-clock parts to the UTC instant they represent.
958
+ * Uses the offset-via-two-formatters trick: guess UTC == the wall-clock parts
959
+ * verbatim, see how far that guess's ET rendering drifts, and correct for it.
960
+ * One correction pass is sufficient since ET offsets only take integer-hour values.
961
+ */
962
+ function etWallClockToUtc(wall) {
963
+ const guessMs = Date.UTC(wall.year, wall.month - 1, wall.day, wall.hour, wall.minute, wall.second);
964
+ const guessAsEt = utcToEtWallClock(new Date(guessMs));
965
+ const driftMs = Date.UTC(guessAsEt.year, guessAsEt.month - 1, guessAsEt.day, guessAsEt.hour, guessAsEt.minute, guessAsEt.second) - guessMs;
966
+ return new Date(guessMs - driftMs);
967
+ }
968
+ /** Weekday of a wall-clock date, Mon=0 ... Sun=6 (matches Python's `date.weekday()`). */
969
+ function weekday(wall) {
970
+ return (new Date(Date.UTC(wall.year, wall.month - 1, wall.day)).getUTCDay() + 6) % 7;
971
+ }
972
+ /** Add (or subtract, for negative n) whole calendar days to a wall-clock moment. */
973
+ function addDays(wall, n) {
974
+ const d = new Date(Date.UTC(wall.year, wall.month - 1, wall.day));
975
+ d.setUTCDate(d.getUTCDate() + n);
976
+ return {
977
+ year: d.getUTCFullYear(),
978
+ month: d.getUTCMonth() + 1,
979
+ day: d.getUTCDate(),
980
+ hour: wall.hour,
981
+ minute: wall.minute,
982
+ second: wall.second
983
+ };
984
+ }
985
+ function atCutoff(wall) {
986
+ return {
987
+ ...wall,
988
+ hour: CUTOFF_HOUR,
989
+ minute: 0,
990
+ second: 0
991
+ };
992
+ }
993
+ function addHours(wall, hours) {
994
+ const totalMinutes = wall.hour * 60 + wall.minute + hours * 60;
995
+ const dayOffset = Math.floor(totalMinutes / 1440);
996
+ const minutesInDay = (totalMinutes % 1440 + 1440) % 1440;
997
+ return {
998
+ ...addDays(wall, dayOffset),
999
+ hour: Math.floor(minutesInDay / 60),
1000
+ minute: minutesInDay % 60,
1001
+ second: wall.second
1002
+ };
1003
+ }
1004
+ function compareWallClock(a, b) {
1005
+ return Date.UTC(a.year, a.month - 1, a.day, a.hour, a.minute, a.second) - Date.UTC(b.year, b.month - 1, b.day, b.hour, b.minute, b.second);
1006
+ }
1007
+ /**
1008
+ * Find the most recent submission window already announced, as of `now`.
1009
+ *
1010
+ * arXiv's daily submission-window cutoff is 14:00 ET; results are announced 20:00 ET
1011
+ * the same day, except Friday's cutoff (which covers the whole weekend) is announced
1012
+ * the following Sunday 20:00. Sat/Sun are never a window's end. A window ending on
1013
+ * Monday spans 3 days (it absorbs the weekend); every other window spans 1 day.
1014
+ *
1015
+ * Returns [start, end] as UTC Dates.
1016
+ */
1017
+ function submissionWindow(now) {
1018
+ const nowEt = utcToEtWallClock(now);
1019
+ let windowEnd = atCutoff(nowEt);
1020
+ if (compareWallClock(nowEt, windowEnd) < 0) windowEnd = addDays(windowEnd, -1);
1021
+ for (;;) {
1022
+ const wd = weekday(windowEnd);
1023
+ if (wd === 5 || wd === 6) {
1024
+ windowEnd = addDays(windowEnd, -1);
1025
+ continue;
1026
+ }
1027
+ if (compareWallClock(nowEt, wd === 4 ? addHours(addDays(windowEnd, 2), 6) : addHours(windowEnd, 6)) >= 0) break;
1028
+ windowEnd = addDays(windowEnd, -1);
1029
+ }
1030
+ const daysBack = weekday(windowEnd) === 0 ? 3 : 1;
1031
+ return [etWallClockToUtc(addDays(windowEnd, -daysBack)), etWallClockToUtc(windowEnd)];
1032
+ }
1033
+ /**
1034
+ * Enumerate the windows to process for `--start-from`/`--windows`, both 1-indexed.
1035
+ * Window #1 is the latest announced window, #2 the one before it, etc.
1036
+ * `startFrom` anchors backward in time; `limit` walks forward (toward more recent)
1037
+ * from the anchor, clamped at #1 (never into the future).
1038
+ *
1039
+ * Returns windows oldest-first.
1040
+ */
1041
+ function windowsToProcess(now, startFrom, limit) {
1042
+ const newestIndex = Math.max(1, startFrom - (limit - 1));
1043
+ const windows = [];
1044
+ let [start, end] = submissionWindow(now);
1045
+ for (let index = 1; index <= startFrom; index++) {
1046
+ if (index >= newestIndex) windows.push([start, end]);
1047
+ [start, end] = submissionWindow(/* @__PURE__ */ new Date(start.getTime() - 1));
1048
+ }
1049
+ return windows.reverse();
1050
+ }
1051
+ //#endregion
1052
+ //#region src/cli.ts
1053
+ const packageVersion = JSON.parse(readFileSync(new URL("../package.json", import.meta.url), "utf-8")).version;
1054
+ const program = new Command();
1055
+ function parsePositiveInteger(value) {
1056
+ if (!/^\d+$/.test(value)) throw new InvalidArgumentError("must be a positive integer");
1057
+ const parsed = Number.parseInt(value, 10);
1058
+ if (!Number.isSafeInteger(parsed) || parsed < 1) throw new InvalidArgumentError("must be a positive integer (1-indexed)");
1059
+ return parsed;
1060
+ }
1061
+ /** Most recently submitted N papers (all of them if `maxPapers` is unset). */
1062
+ function capMostRecent(papers, maxPapers) {
1063
+ if (maxPapers === void 0 || papers.length <= maxPapers) return papers;
1064
+ return [...papers].sort((a, b) => b.published.localeCompare(a.published)).slice(0, maxPapers);
1065
+ }
1066
+ function formatProgress(stage, p) {
1067
+ const stageName = stage === "coarse" ? "Coarse" : "Fine";
1068
+ const detail = stage === "coarse" ? `${p.passed}/${p.papersTotal} papers kept, ` : "";
1069
+ return `${stageName} filtering: ${p.papersDone}/${p.papersTotal} papers processed (${detail}${p.callsDone}/${p.callsTotal} calls done, ${p.failed} failed)`;
1070
+ }
1071
+ /** Progress snapshot for a stage that had nothing pending (all papers already screened/scored). */
1072
+ function alreadyDoneProgress(papers, stage) {
1073
+ const passed = stage === "coarse" ? papers.filter((p) => p.coarse === 1).length : papers.filter((p) => p.score !== void 0).length;
1074
+ return {
1075
+ papersDone: papers.length,
1076
+ papersTotal: papers.length,
1077
+ passed,
1078
+ callsDone: 0,
1079
+ callsTotal: 0,
1080
+ failed: 0
1081
+ };
1082
+ }
1083
+ function renderDigestHtml(subject, body) {
1084
+ return `<!doctype html>
1085
+ <html>
1086
+ <head><meta charset="utf-8"><title>${subject}</title></head>
1087
+ <body>
1088
+ ${body}
1089
+ </body>
1090
+ </html>
1091
+ `;
1092
+ }
1093
+ program.name("paperino").description("Every new arXiv paper, every day, filtered by Claude Code down to what's actually worth your time.").version(packageVersion).option("--configure", `open ${CONFIG_PATH} in your default editor.`, false).option("--email", "get the digest directly to your inbox (coming soon).", false).option("--force", "discard any existing run/digest for the given windows and rerun from scratch.", false).option("--logs", "page through the log file with less; no pipeline run.", false).option("--max-papers <n>", "cap papers processed per window, most recently submitted first (default: unlimited).", parsePositiveInteger).option("--only-fetch", "only fetch the papers from arXiv. Do NOT run the relevance-scoring pipeline.", false).option("--quiet", "suppress progress output; only print the path to the output file.", false).option("--start-from <n>", "1-indexed window to start from, most recent first .", parsePositiveInteger, 1).option("--windows <n>", "number of windows to process, walking forward from --start-from toward the present.", parsePositiveInteger, 1).option("-y, --yes", "skip the confirmation prompt and run immediately.", false).action(async (options) => {
1094
+ if (options.email) throw new Error("email delivery is not implemented yet");
1095
+ ensureConfig();
1096
+ if (options.configure) {
1097
+ openConfigInEditor();
1098
+ return;
1099
+ }
1100
+ const cfg = loadConfig();
1101
+ if (options.logs) {
1102
+ viewLogs(cfg.logFile);
1103
+ return;
1104
+ }
1105
+ const logger = makeLogger(cfg.logFile);
1106
+ const windows = windowsToProcess(/* @__PURE__ */ new Date(), options.startFrom, options.windows);
1107
+ const maxPapers = options.maxPapers;
1108
+ if (!options.yes && !await confirmRun(windows, cfg, maxPapers, options.force ?? false, options.onlyFetch ?? false)) {
1109
+ process.stderr.write(pc.dim("aborted.\n"));
1110
+ return;
1111
+ }
1112
+ const agent = new ClaudeAgent(cfg.claudeBinary);
1113
+ const outputPaths = [];
1114
+ let hadFailures = false;
1115
+ logger.pipelineStart(windows.length);
1116
+ for (const [start, end] of windows) {
1117
+ if (options.force) clearRunDir(cfg.outDir, end);
1118
+ const dir = runDir(cfg.outDir, end);
1119
+ const label = formatUTCDate(end);
1120
+ logger.windowStart(label);
1121
+ const view = options.quiet ? makeSilentPipelineView() : makePipelineView(label, options.onlyFetch ? ["Fetching papers"] : void 0);
1122
+ logger.stageStart("Fetching papers");
1123
+ const existing = loadPapers(dir);
1124
+ const fetched = await fetchRecentPapers(cfg.arxivCat, start, end, maxPapers);
1125
+ const papers = mergePapers(existing, fetched);
1126
+ savePapers(dir, papers);
1127
+ const toProcess = capMostRecent(papers, maxPapers);
1128
+ const cappedNote = maxPapers !== void 0 && toProcess.length < papers.length ? ` (only scoring most recent ${toProcess.length})` : "";
1129
+ const fetchMetrics = `Fetched ${fetched.length} papers, ${papers.length} total in store${cappedNote}`;
1130
+ view.complete(fetchMetrics);
1131
+ logger.stageEnd("Fetching papers", fetchMetrics);
1132
+ if (options.onlyFetch) {
1133
+ view.stop();
1134
+ outputPaths.push(`${dir}/papers.json`);
1135
+ continue;
1136
+ }
1137
+ logger.stageStart("Coarse filtering");
1138
+ let lastCoarse;
1139
+ await coarseFilter(toProcess, cfg, agent, (p) => {
1140
+ lastCoarse = p;
1141
+ view.update(formatProgress("coarse", p));
1142
+ }, (err) => logger.callFailed("Coarse filtering", err));
1143
+ const coarseProgress = lastCoarse ?? alreadyDoneProgress(toProcess, "coarse");
1144
+ const coarseMetrics = formatProgress("coarse", coarseProgress);
1145
+ view.complete(coarseMetrics, coarseProgress.failed > 0);
1146
+ logger.stageEnd("Coarse filtering", coarseMetrics);
1147
+ savePapers(dir, papers);
1148
+ logger.stageStart("Fine filtering");
1149
+ let lastFine;
1150
+ await fineScoring(toProcess, cfg, agent, (p) => {
1151
+ lastFine = p;
1152
+ view.update(formatProgress("fine", p));
1153
+ }, (err) => logger.callFailed("Fine filtering", err));
1154
+ const fineProgress = lastFine ?? alreadyDoneProgress(toProcess, "fine");
1155
+ const fineMetrics = formatProgress("fine", fineProgress);
1156
+ view.complete(fineMetrics, fineProgress.failed > 0);
1157
+ logger.stageEnd("Fine filtering", fineMetrics);
1158
+ savePapers(dir, papers);
1159
+ view.stop();
1160
+ const { subject, body } = buildDigest(papers, label, cfg.minScore);
1161
+ const htmlPath = writeDigest(dir, renderDigestHtml(subject, body));
1162
+ if (coarseProgress.failed > 0 || fineProgress.failed > 0) {
1163
+ hadFailures = true;
1164
+ const failedCalls = coarseProgress.failed + fineProgress.failed;
1165
+ if (!options.quiet) process.stderr.write(`${pc.yellow("⚠ Done with errors.")} ${pc.dim(`Digest may be incomplete: ${failedCalls} call(s) failed. Run`)} paperino --logs ${pc.dim("for details.")}\n${pc.dim("Digest ready at")} ${htmlPath}\n\n`);
1166
+ } else if (!options.quiet) process.stderr.write(`${pc.green("Done.")} ${pc.dim("Digest ready at")} ${htmlPath}\n\n`);
1167
+ outputPaths.push(htmlPath);
1168
+ }
1169
+ logger.pipelineEnd();
1170
+ if (options.quiet) for (const path of outputPaths) process.stdout.write(`${path}\n`);
1171
+ if (hadFailures) process.exitCode = 1;
1172
+ });
1173
+ program.parseAsync().catch((err) => {
1174
+ const message = err instanceof Error ? err.message : String(err);
1175
+ process.stderr.write(`error: ${message}\n`);
1176
+ process.exitCode = 1;
1177
+ });
1178
+ //#endregion
1179
+ export {};
package/package.json ADDED
@@ -0,0 +1,45 @@
1
+ {
2
+ "name": "@giacomo-ciro/paperino",
3
+ "version": "0.1.0",
4
+ "description": "",
5
+ "author": "giacomo-ciro",
6
+ "license": "MIT",
7
+ "type": "module",
8
+ "bin": {
9
+ "paperino": "dist/cli.mjs"
10
+ },
11
+ "engines": {
12
+ "node": ">=24"
13
+ },
14
+ "files": [
15
+ "dist"
16
+ ],
17
+ "scripts": {
18
+ "start": "node dist/cli.mjs",
19
+ "build": "tsdown",
20
+ "dev": "tsdown --watch",
21
+ "test": "vitest run"
22
+ },
23
+ "dependencies": {
24
+ "cli-spinners": "^3.4.0",
25
+ "commander": "^15.0.0",
26
+ "fast-xml-parser": "^5.10.1",
27
+ "ora": "^9.4.1",
28
+ "picocolors": "^1.1.1",
29
+ "smol-toml": "^1.3.1"
30
+ },
31
+ "devDependencies": {
32
+ "@types/node": "^26.1.1",
33
+ "tsdown": "^0.22.9",
34
+ "tsx": "^4.23.1",
35
+ "typescript": "^7.0.2",
36
+ "vitest": "^3.2.4"
37
+ },
38
+ "repository": {
39
+ "type": "git",
40
+ "url": "https://github.com/giacomo-ciro/paperino"
41
+ },
42
+ "publishConfig": {
43
+ "access": "public"
44
+ }
45
+ }