@tiagojct/subsub 0.17.0 → 0.18.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +14 -0
- package/README.md +1 -0
- package/dist/config.js +1 -1
- package/dist/feedback.js +55 -0
- package/dist/notes.js +41 -11
- package/dist/read.js +2 -1
- package/dist/subsub.js +80 -6
- package/dist/web.js +22 -5
- package/package.json +1 -1
- package/prompts/compare.md +1 -1
- package/prompts/gaps.md +1 -1
- package/prompts/lit.md +26 -7
- package/roles/researcher.md +5 -2
- package/src/config.ts +1 -1
- package/src/feedback.ts +70 -0
- package/src/notes.ts +44 -10
- package/src/read.ts +2 -1
- package/src/subsub.ts +83 -9
- package/src/web.ts +20 -5
- package/templates/Zotero agent.md +1 -1
- package/web/app.css +11 -0
- package/web/app.js +82 -5
- package/web/i18n.js +6 -1
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,20 @@
|
|
|
2
2
|
|
|
3
3
|
Sub-Sub follows the version numbers of npm: a change in the second number (0.13) adds features or changes how you use it; a change in the third (0.13.1) only fixes.
|
|
4
4
|
|
|
5
|
+
## 0.18.0 (2026-10-15)
|
|
6
|
+
|
|
7
|
+
- Figures and tables from the PDF. With the Zotero server 0.6.0, Sub-Sub can cut a figure or a table out of an item's PDF (`zotero_get_figure`, with the label of its caption, such as "Figure 2" or "Table 1") and show it in a note, with its caption and a link to the page. `/read critique` shows the table or figure with the main result. The images are PNG files in `Literature/figures/`; the notes pane and Obsidian show them. When the figure cannot be cut out, Sub-Sub takes the whole page and says so.
|
|
8
|
+
- In a note written from the full text, the page after each quote, "(p. N)", becomes a link that opens the PDF in Zotero at the page where Sub-Sub found the quote. The label stays as written; the link goes to the right page even when the printed page number differs from the PDF's.
|
|
9
|
+
- `/feedback [what happened]` (Report a problem in the browser) writes a draft bug report in `Inbox/`: the versions, the mode and model, the output of `subsub doctor` and the last error, with your folders, email addresses and keys taken out. You read it and post it on GitHub; Sub-Sub sends nothing.
|
|
10
|
+
- Preprints (medRxiv, bioRxiv, Research Square, SSRN, arXiv) are marked as not peer reviewed wherever Sub-Sub uses them: "(preprint)" in evidence tables and comparison matrices, and in the text. When a journal version exists, Sub-Sub uses and cites it instead.
|
|
11
|
+
- `/lit` rates how hard the search is (1 to 10) and allows none, one or two follow-up search rounds to match, with one publication window for every search; it stops early when the sources cover the question. The note's Methods give the rating and the rounds. `/gaps` gives a ranked list of 5 to 15 works and does not read them. This keeps the cost of a search in proportion to the question.
|
|
12
|
+
- In the browser view, every table in a note or a reply has a Download CSV button: evidence tables, extraction sheets, comparison matrices. The file opens in Excel (with semicolons when the interface is in Portuguese, as Excel expects there), Rayyan or Covidence.
|
|
13
|
+
- A `/lit` note (and a reading list) shows the flow of works under Methods: found, unique, screened, read and used, with what was left out at each step. It is a Mermaid diagram, so Obsidian, Quarto and GitHub draw it; the browser view draws it too.
|
|
14
|
+
- The quote check is stricter. A quote may skip text only at a page break (the running footer and header); two passages of one page joined into one quote, without [...] for the omission, are refused. Zotero marks page breaks in the full text, so the check knows where they are.
|
|
15
|
+
- `/lit reading list:` no longer ends with "the answer in three lines": a reading list states no results, in the reply too.
|
|
16
|
+
- In Portuguese, the browser button for `/lit-note` is now Nota de literatura, so that it is not confused with the reading notes of `/read`.
|
|
17
|
+
- Delete in the browser view keeps the conversation's subfolder in `~/.subsub/trash`, so that moving it back is one step.
|
|
18
|
+
|
|
5
19
|
## 0.17.0 (2026-10-14)
|
|
6
20
|
|
|
7
21
|
For people who are learning to read the literature:
|
package/README.md
CHANGED
|
@@ -95,6 +95,7 @@ Commands in Sub-Sub:
|
|
|
95
95
|
| `/researcher`, `/librarian` | Switch the mode |
|
|
96
96
|
| `/subsub` | Zotero connection, mode, model and library counts |
|
|
97
97
|
| `/history`, `/undo [id]` | Recent library changes; undo the last one or one by its id |
|
|
98
|
+
| `/feedback [what happened]` | A draft bug report in `Inbox/`, with versions, `subsub doctor` and the last error, and without your folders, emails or keys. You post it on GitHub; Sub-Sub sends nothing |
|
|
98
99
|
| `/ai-statement [uses]` | A draft statement on the use of AI for a manuscript, with the Sub-Sub version, its DOI and your models |
|
|
99
100
|
| `/tag-batch [size]` | Tag items that have no topic, with your tag list |
|
|
100
101
|
| `/clean-tags` | Find tags that are not in your tag list and propose fixes |
|
package/dist/config.js
CHANGED
|
@@ -57,7 +57,7 @@ export function uiLanguage(language) {
|
|
|
57
57
|
return /portug|^pt\b|^pt-/i.test(language ?? "") ? "pt" : "en";
|
|
58
58
|
}
|
|
59
59
|
/** The Zotero server release that this Sub-Sub version runs when there is no local server folder. */
|
|
60
|
-
export const SERVER_VERSION = "0.
|
|
60
|
+
export const SERVER_VERSION = "0.6.0";
|
|
61
61
|
/** The Starbuck release (PyPI) that this Sub-Sub version runs when there is no local Starbuck folder. */
|
|
62
62
|
export const STARBUCK_VERSION = "0.2.0";
|
|
63
63
|
export const STARBUCK_SOURCE = `starbuck==${STARBUCK_VERSION}`;
|
package/dist/feedback.js
ADDED
|
@@ -0,0 +1,55 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* /feedback: a draft bug report or idea in the Inbox, for the user to read and post on GitHub.
|
|
3
|
+
* Sub-Sub sends nothing: the user decides what to share. The home and Sub-Sub folders, email
|
|
4
|
+
* addresses and keys are taken out before the draft is written.
|
|
5
|
+
*/
|
|
6
|
+
export const ISSUES_URL = "https://github.com/tiagojct/subsub/issues/new";
|
|
7
|
+
/** Text made safe to post: the Sub-Sub and home folders, email addresses and keys taken out. */
|
|
8
|
+
export function redact(text, home, vault) {
|
|
9
|
+
let out = text;
|
|
10
|
+
if (vault)
|
|
11
|
+
out = out.split(vault).join("<Sub-Sub folder>");
|
|
12
|
+
if (home)
|
|
13
|
+
out = out.split(home).join("~");
|
|
14
|
+
return out
|
|
15
|
+
.replace(/[\w.+-]+@[\w-]+(?:\.[\w-]+)+/g, "<email>")
|
|
16
|
+
.replace(/\b(?:sk-|AIza|ghp_|gho_|github_pat_|xox[bp]-)[\w-]{8,}/g, "<key>")
|
|
17
|
+
.replace(/\b[A-Za-z0-9_-]{32,}\b/g, "<key>");
|
|
18
|
+
}
|
|
19
|
+
export function feedbackNote(f) {
|
|
20
|
+
const clean = (s) => redact(s, f.home, f.vault);
|
|
21
|
+
return [
|
|
22
|
+
"---",
|
|
23
|
+
"type: feedback",
|
|
24
|
+
`created: ${f.date}`,
|
|
25
|
+
"---",
|
|
26
|
+
"",
|
|
27
|
+
"# Sub-Sub feedback",
|
|
28
|
+
"",
|
|
29
|
+
`Read this draft, take out anything you do not want to share, then post it at ${ISSUES_URL}. Sub-Sub has sent nothing.`,
|
|
30
|
+
"",
|
|
31
|
+
"## What happened",
|
|
32
|
+
"",
|
|
33
|
+
clean(f.what.trim()) || "(what you did, and what you saw)",
|
|
34
|
+
"",
|
|
35
|
+
"## What you expected",
|
|
36
|
+
"",
|
|
37
|
+
"(what should have happened)",
|
|
38
|
+
"",
|
|
39
|
+
"## Set-up",
|
|
40
|
+
"",
|
|
41
|
+
`- ${clean(f.versions)}`,
|
|
42
|
+
...f.setup.map((l) => `- ${clean(l)}`),
|
|
43
|
+
"",
|
|
44
|
+
"## subsub doctor",
|
|
45
|
+
"",
|
|
46
|
+
"```",
|
|
47
|
+
clean(f.checks.trim()),
|
|
48
|
+
"```",
|
|
49
|
+
"",
|
|
50
|
+
"## Last error",
|
|
51
|
+
"",
|
|
52
|
+
f.lastError ? ["```", clean(f.lastError.trim()).slice(0, 2000), "```"].join("\n") : "(none in this session)",
|
|
53
|
+
"",
|
|
54
|
+
].join("\n");
|
|
55
|
+
}
|
package/dist/notes.js
CHANGED
|
@@ -77,12 +77,16 @@ export function matchable(s) {
|
|
|
77
77
|
.toLowerCase()
|
|
78
78
|
.replace(/[^\p{L}\p{N}]+/gu, "");
|
|
79
79
|
}
|
|
80
|
+
/** The full text as matchable pages joined by "|": Zotero marks a page break with a form feed. */
|
|
81
|
+
export function sourceOf(fullText) {
|
|
82
|
+
return fullText.split("\f").map(matchable).join("|");
|
|
83
|
+
}
|
|
80
84
|
/**
|
|
81
85
|
* The quotes that are not in the full text. A quote with an ellipsis is checked in parts;
|
|
82
86
|
* square brackets (an editor's insertion) are left out of the comparison.
|
|
83
87
|
*/
|
|
84
88
|
export function missingQuotes(note, fullText, already = []) {
|
|
85
|
-
const source =
|
|
89
|
+
const source = sourceOf(fullText);
|
|
86
90
|
const old = new Set(already);
|
|
87
91
|
return quotesIn(note)
|
|
88
92
|
.filter((q) => !old.has(q))
|
|
@@ -92,33 +96,59 @@ export function missingQuotes(note, fullText, already = []) {
|
|
|
92
96
|
});
|
|
93
97
|
}
|
|
94
98
|
/**
|
|
95
|
-
*
|
|
96
|
-
*
|
|
99
|
+
* Where a passage starts in the source (from sourceOf), or -1. It must be there word for word.
|
|
100
|
+
* The only gap allowed is a page break with the running footer and header around it (groups of
|
|
101
|
+
* three words in order, at most `gap` characters apart), so that a quote cannot join two passages
|
|
102
|
+
* of one page without marking the omission.
|
|
97
103
|
*/
|
|
98
|
-
export function
|
|
99
|
-
|
|
100
|
-
|
|
104
|
+
export function locate(source, passage, gap = 300) {
|
|
105
|
+
const whole = source.indexOf(matchable(passage));
|
|
106
|
+
if (whole >= 0)
|
|
107
|
+
return whole;
|
|
101
108
|
const words = passage.split(/\s+/).map(matchable).filter(Boolean);
|
|
102
109
|
const groups = [];
|
|
103
110
|
for (let i = 0; i < words.length; i += 3)
|
|
104
111
|
groups.push(words.slice(i, i + 3).join(""));
|
|
105
112
|
if (groups.length < 2)
|
|
106
|
-
return
|
|
113
|
+
return -1;
|
|
107
114
|
for (let start = source.indexOf(groups[0]), tries = 0; start >= 0 && tries < 50; start = source.indexOf(groups[0], start + 1), tries++) {
|
|
108
115
|
let end = start + groups[0].length;
|
|
109
116
|
let ok = true;
|
|
110
117
|
for (const g of groups.slice(1)) {
|
|
111
118
|
const at = source.indexOf(g, end);
|
|
112
|
-
if (at < 0 || at - end > gap) {
|
|
119
|
+
if (at < 0 || (at > end && (at - end > gap || !source.slice(end, at).includes("|")))) {
|
|
113
120
|
ok = false;
|
|
114
121
|
break;
|
|
115
122
|
}
|
|
116
123
|
end = at + g.length;
|
|
117
124
|
}
|
|
118
125
|
if (ok)
|
|
119
|
-
return
|
|
126
|
+
return start;
|
|
120
127
|
}
|
|
121
|
-
return
|
|
128
|
+
return -1;
|
|
129
|
+
}
|
|
130
|
+
export function appearsIn(source, passage, gap = 300) {
|
|
131
|
+
return locate(source, passage, gap) >= 0;
|
|
132
|
+
}
|
|
133
|
+
/** The page of the PDF (1-based, as Zotero's reader counts) where a passage starts, or undefined. */
|
|
134
|
+
export function pageOf(source, passage) {
|
|
135
|
+
const at = locate(source, passage);
|
|
136
|
+
return at < 0 ? undefined : source.slice(0, at).split("|").length;
|
|
137
|
+
}
|
|
138
|
+
/**
|
|
139
|
+
* Each "(p. N)" after a quote becomes a link that opens the PDF in Zotero at the page where the quote
|
|
140
|
+
* is. The label stays as written (the page printed on the paper may differ from the PDF's count).
|
|
141
|
+
*/
|
|
142
|
+
export function withPageLinks(text, source, attachment) {
|
|
143
|
+
const link = (quote, label) => {
|
|
144
|
+
const page = pageOf(source, quote.split(/\s*(?:\.\.\.|…|\[\s*(?:\.\.\.|…)\s*\])\s*/)[0]);
|
|
145
|
+
return page ? `([${label}](zotero://open-pdf/library/items/${attachment}?page=${page}))` : `(${label})`;
|
|
146
|
+
};
|
|
147
|
+
const PAGE = String.raw `\((pp?\.\s?\d+(?:\s?[-–]\s?\d+)?)\)`;
|
|
148
|
+
return text
|
|
149
|
+
.replace(new RegExp(String.raw `(")([^"\n]{25,600})("\s*)${PAGE}`, "g"), (_m, o, q, c, label) => `${o}${q}${c}${link(q, label)}`)
|
|
150
|
+
.replace(new RegExp(String.raw `(“)([^“”\n]{25,600})(”\s*)${PAGE}`, "g"), (_m, o, q, c, label) => `${o}${q}${c}${link(q, label)}`)
|
|
151
|
+
.replace(new RegExp(String.raw `^(>[ \t]*)(.{25,}?)(\s*)${PAGE}[ \t]*$`, "gm"), (_m, o, q, c, label) => `${o}${q}${c}${link(q.replace(/^["“]|["”]$/g, ""), label)}`);
|
|
122
152
|
}
|
|
123
153
|
export function quoteProblem(missing) {
|
|
124
154
|
if (!missing.length)
|
|
@@ -127,6 +157,6 @@ export function quoteProblem(missing) {
|
|
|
127
157
|
return [
|
|
128
158
|
`${missing.length} quote(s) in this note are not in the item's full text:`,
|
|
129
159
|
...shown,
|
|
130
|
-
"Copy each quote word for word from the full text (zotero_get_fulltext)
|
|
160
|
+
"Copy each quote word for word from the full text (zotero_get_fulltext). Mark every omission with [...]: two passages joined without that mark are not a quote. Or remove the quote marks and say it in your own words. Then write the note again.",
|
|
131
161
|
].join("\n");
|
|
132
162
|
}
|
package/dist/read.js
CHANGED
|
@@ -59,6 +59,7 @@ function depth(r) {
|
|
|
59
59
|
"Depth: the whole paper, section by section. A critique needs the full text: if you can read only the abstract, say so, write what the abstract allows, and offer /request-copy to ask the authors for the paper.",
|
|
60
60
|
'Under "## Methods", "## Results" and "## Discussion": the passages that matter to judge the study, quoted with their place, each with one question for the user (for example: how were the participants selected? what was compared? what else could explain the result?).',
|
|
61
61
|
`Under "## Appraisal": name the design and the reporting guideline that applies (${GUIDELINES}), and tell the user to check the paper against it item by item. Do not copy the checklist. Then five to eight questions on the risk of bias for this design (selection, confounding, measurement, missing data, selective reporting), each with the passage to look at.`,
|
|
62
|
+
"If the zotero_get_figure tool is available, show the table or figure with the main result: call it with the label of its caption and put the Markdown line it returns under the passage that cites it, with the page link, and a question about what the user sees in it.",
|
|
62
63
|
];
|
|
63
64
|
}
|
|
64
65
|
}
|
|
@@ -79,7 +80,7 @@ export function readInstruction(r) {
|
|
|
79
80
|
"",
|
|
80
81
|
"Rules:",
|
|
81
82
|
"- State no results, effect sizes, summaries or conclusions, in the note or in the chat. Quote the paper and ask; the user writes the answers.",
|
|
82
|
-
"- Copy quotes word for word from what you read. After each quote give its place:
|
|
83
|
+
"- Copy quotes word for word from what you read, and mark every omission with [...]. After each quote give its place: (p. N) only with evidence: full text (Sub-Sub links it to that page of the PDF), otherwise the section (Abstract, Methods, Results).",
|
|
83
84
|
`- The note is Literature/<citekey>.md in the Sub-Sub folder, with the front matter of a literature note from the shared rules plus type: reading-note and purpose: ${r.purpose}; evidence says what you read. If the note exists, do not overwrite it: add a section "## Reading ${r.date}: ${r.purpose}" at the end with edit, with the content below one heading level lower.`,
|
|
84
85
|
'- Start with "## Why I am reading this" (the reason, in the user\'s words when they gave them) and end with one line "Next step:" (what to read or do next).',
|
|
85
86
|
"",
|
package/dist/subsub.js
CHANGED
|
@@ -14,14 +14,17 @@
|
|
|
14
14
|
* session_shutdown. Tools are registered once and use the current bridge.
|
|
15
15
|
*/
|
|
16
16
|
import { existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
|
|
17
|
+
import { homedir } from "node:os";
|
|
17
18
|
import { dirname, join } from "node:path";
|
|
18
19
|
import { fileURLToPath } from "node:url";
|
|
19
20
|
import { Bridge } from "./bridge.js";
|
|
20
21
|
import { expand, findUv, FREE_MODELS, loadConfig, parseEnvFile, serverCommand, starbuckCommand, starbuckEnabled, uiLanguage, who, } from "./config.js";
|
|
21
22
|
import { coerceArgs, loosenArrays, resolvePathArgs } from "./args.js";
|
|
22
23
|
import { aiStatement } from "./statement.js";
|
|
24
|
+
import { feedbackNote, ISSUES_URL } from "./feedback.js";
|
|
25
|
+
import { formatChecks, runChecks } from "./doctor.js";
|
|
23
26
|
import { PURPOSES, readArgs, readInstruction } from "./read.js";
|
|
24
|
-
import { applyEdits, checkLiteratureNote, field, frontMatter, itemKeyOf, missingQuotes, quoteProblem, quotesIn } from "./notes.js";
|
|
27
|
+
import { applyEdits, checkLiteratureNote, field, frontMatter, itemKeyOf, missingQuotes, quoteProblem, quotesIn, sourceOf, withPageLinks } from "./notes.js";
|
|
25
28
|
import { buildPolicy, isSecret, judgePath, normalizeToolPath, realish, within } from "./paths.js";
|
|
26
29
|
import { describeArgs, formatPreview, isEmptyPreview } from "./preview.js";
|
|
27
30
|
import { perMinuteWait, providerTrouble, TROUBLE_TEXT, TROUBLE_TEXT_PT } from "./keys.js";
|
|
@@ -127,6 +130,8 @@ export async function createSubsub(pi, deps = {}) {
|
|
|
127
130
|
/** True under `subsub web`: pi runs in RPC mode and the web view shows the state that the terminal header shows. */
|
|
128
131
|
const webView = process.env.SUBSUB_WEB === "1";
|
|
129
132
|
let lastError;
|
|
133
|
+
/** The last provider or tool error of the session, for /feedback. */
|
|
134
|
+
let recentError;
|
|
130
135
|
let resumes = 0;
|
|
131
136
|
let resumeTimer;
|
|
132
137
|
let turnStart = 0;
|
|
@@ -319,13 +324,19 @@ export async function createSubsub(pi, deps = {}) {
|
|
|
319
324
|
const m = event.message;
|
|
320
325
|
if (m.role !== "assistant" || m.stopReason !== "error")
|
|
321
326
|
return;
|
|
322
|
-
lastError = m.errorMessage;
|
|
327
|
+
lastError = recentError = m.errorMessage;
|
|
323
328
|
if (webView || perMinuteWait(m.errorMessage) !== undefined)
|
|
324
329
|
return;
|
|
325
330
|
const kind = providerTrouble(m.errorMessage);
|
|
326
331
|
if (kind)
|
|
327
332
|
ctx.ui.notify(`Sub-Sub: ${trouble[kind]}`, "warning");
|
|
328
333
|
});
|
|
334
|
+
pi.on("tool_result", async (event) => {
|
|
335
|
+
if (!event.isError)
|
|
336
|
+
return;
|
|
337
|
+
const text = event.content.map((c) => (c.type === "text" ? c.text : "")).join(" ");
|
|
338
|
+
recentError = `${event.toolName ?? "tool"}: ${text}`.slice(0, 2000);
|
|
339
|
+
});
|
|
329
340
|
// A per-minute limit (free tiers): wait as long as the provider says, then continue the task,
|
|
330
341
|
// up to three times per request of the user. Anything the user types cancels the wait.
|
|
331
342
|
pi.on("agent_end", async (_event, ctx) => {
|
|
@@ -507,14 +518,28 @@ export async function createSubsub(pi, deps = {}) {
|
|
|
507
518
|
return undefined;
|
|
508
519
|
const full = await fullTextOf(key, signal);
|
|
509
520
|
// Without the full text (no PDF indexed, Zotero closed) there is nothing to check against.
|
|
510
|
-
|
|
521
|
+
if (!full)
|
|
522
|
+
return undefined;
|
|
523
|
+
const missing = quoteProblem(missingQuotes(text, full.text, quotesIn(before)));
|
|
524
|
+
// A note that passes is written with its page references linked to the PDF.
|
|
525
|
+
if (!missing && full.attachment) {
|
|
526
|
+
const link = (s) => withPageLinks(s, sourceOf(full.text), full.attachment);
|
|
527
|
+
if (name === "write")
|
|
528
|
+
input.content = link(String(input.content ?? ""));
|
|
529
|
+
else
|
|
530
|
+
for (const e of input.edits)
|
|
531
|
+
if (typeof e.newText === "string")
|
|
532
|
+
e.newText = link(e.newText);
|
|
533
|
+
}
|
|
534
|
+
return missing;
|
|
511
535
|
}
|
|
512
|
-
/** The whole indexed full text of an item, read in parts, kept for the session. */
|
|
536
|
+
/** The whole indexed full text of an item and its attachment, read in parts, kept for the session. */
|
|
513
537
|
const fullTexts = new Map();
|
|
514
538
|
async function fullTextOf(key, signal) {
|
|
515
539
|
if (fullTexts.has(key))
|
|
516
540
|
return fullTexts.get(key) ?? undefined;
|
|
517
541
|
let text = "";
|
|
542
|
+
let attachment;
|
|
518
543
|
let offset = 0;
|
|
519
544
|
try {
|
|
520
545
|
for (let part = 0; offset !== null && part < 20; part++) {
|
|
@@ -523,14 +548,16 @@ export async function createSubsub(pi, deps = {}) {
|
|
|
523
548
|
break;
|
|
524
549
|
const data = (res.data ?? {});
|
|
525
550
|
text += data.text ?? "";
|
|
551
|
+
attachment ??= data.attachment;
|
|
526
552
|
offset = data.next_offset ?? null;
|
|
527
553
|
}
|
|
528
554
|
}
|
|
529
555
|
catch {
|
|
530
556
|
/* no full text: the check is skipped */
|
|
531
557
|
}
|
|
532
|
-
|
|
533
|
-
|
|
558
|
+
const full = text ? { text, attachment } : null;
|
|
559
|
+
fullTexts.set(key, full);
|
|
560
|
+
return full ?? undefined;
|
|
534
561
|
}
|
|
535
562
|
/**
|
|
536
563
|
* Starbuck writes its reports into a folder: report_dir, or _starbuck next to the
|
|
@@ -728,6 +755,53 @@ export async function createSubsub(pi, deps = {}) {
|
|
|
728
755
|
pi.sendUserMessage(readInstruction(req), ctx.isIdle() ? undefined : { deliverAs: "followUp" });
|
|
729
756
|
},
|
|
730
757
|
});
|
|
758
|
+
pi.registerCommand("feedback", {
|
|
759
|
+
description: "Sub-Sub: a draft bug report or idea in Inbox/, to read and post on GitHub yourself; Sub-Sub sends nothing [what happened]",
|
|
760
|
+
handler: async (args, ctx) => {
|
|
761
|
+
const date = new Date().toISOString().slice(0, 10);
|
|
762
|
+
let piVersion = "";
|
|
763
|
+
try {
|
|
764
|
+
piVersion = JSON.parse(readFileSync(join(dirname(fileURLToPath(import.meta.resolve("@earendil-works/pi-coding-agent"))), "..", "package.json"), "utf8")).version;
|
|
765
|
+
}
|
|
766
|
+
catch {
|
|
767
|
+
/* not needed for the report */
|
|
768
|
+
}
|
|
769
|
+
let checks;
|
|
770
|
+
try {
|
|
771
|
+
checks = deps.doctor ? deps.doctor() : formatChecks(runChecks(cfg, process.env));
|
|
772
|
+
}
|
|
773
|
+
catch (err) {
|
|
774
|
+
checks = `subsub doctor failed: ${err.message}`;
|
|
775
|
+
}
|
|
776
|
+
const text = feedbackNote({
|
|
777
|
+
what: String(args ?? ""),
|
|
778
|
+
date,
|
|
779
|
+
versions: `Sub-Sub ${ownVersion()}${piVersion ? `, pi ${piVersion}` : ""}, Node ${process.version}, ${process.platform} ${process.arch}`,
|
|
780
|
+
setup: [
|
|
781
|
+
`Mode: ${mode}. Model: ${ctx.model ? `${ctx.model.provider}/${ctx.model.id}` : "none"}.`,
|
|
782
|
+
`Language: ${cfg.language}. Reference checks (Starbuck): ${starbuckEnabled(cfg) ? "on" : "off"}.`,
|
|
783
|
+
`Zotero: ${library.zotero}.${bridge && Object.keys(bridge.errors).length ? ` Servers not running: ${Object.keys(bridge.errors).join(", ")}.` : ""}`,
|
|
784
|
+
],
|
|
785
|
+
checks,
|
|
786
|
+
lastError: recentError,
|
|
787
|
+
home: homedir(),
|
|
788
|
+
vault: cfg.vault,
|
|
789
|
+
});
|
|
790
|
+
const dir = join(cfg.vault ?? ctx.cwd, "Inbox");
|
|
791
|
+
let file = join(dir, `${date} Sub-Sub feedback.md`);
|
|
792
|
+
for (let n = 2; existsSync(file); n++)
|
|
793
|
+
file = join(dir, `${date} Sub-Sub feedback-${n}.md`);
|
|
794
|
+
try {
|
|
795
|
+
mkdirSync(dir, { recursive: true });
|
|
796
|
+
writeFileSync(file, text);
|
|
797
|
+
}
|
|
798
|
+
catch (err) {
|
|
799
|
+
ctx.ui.notify(`${L("Could not save it", "Não foi possível guardar")}: ${err.message}`, "warning");
|
|
800
|
+
return;
|
|
801
|
+
}
|
|
802
|
+
ctx.ui.notify(L(`Sub-Sub: a draft report is in ${file}. Read it, add what you expected, take out anything you do not want to share, and post it at ${ISSUES_URL}. Sub-Sub has sent nothing.`, `Sub-Sub: há um rascunho do relatório em ${file}. Leia-o, acrescente o que esperava, retire o que não quer partilhar e publique-o em ${ISSUES_URL}. O Sub-Sub não enviou nada.`), "info");
|
|
803
|
+
},
|
|
804
|
+
});
|
|
731
805
|
pi.registerCommand("history", {
|
|
732
806
|
description: "Sub-Sub: recent library changes",
|
|
733
807
|
handler: async (_args, ctx) => {
|
package/dist/web.js
CHANGED
|
@@ -18,7 +18,7 @@ import { randomBytes, timingSafeEqual } from "node:crypto";
|
|
|
18
18
|
import { createReadStream, existsSync, mkdirSync, readdirSync, readFileSync, renameSync, statSync, writeFileSync, appendFileSync } from "node:fs";
|
|
19
19
|
import { createServer } from "node:http";
|
|
20
20
|
import { homedir } from "node:os";
|
|
21
|
-
import { basename, dirname, extname, join, normalize, resolve, sep } from "node:path";
|
|
21
|
+
import { basename, dirname, extname, join, normalize, relative, resolve, sep } from "node:path";
|
|
22
22
|
import { StringDecoder } from "node:string_decoder";
|
|
23
23
|
import { agentDirFor, chooseCwd, PACKAGE_DIR, packageVersion } from "./cli.js";
|
|
24
24
|
import { configPath, expand, loadConfig, RECOMMENDED_MODELS, readConfigFile, saveConfig, uiLanguage, withStarbuck } from "./config.js";
|
|
@@ -184,6 +184,8 @@ const TYPES = {
|
|
|
184
184
|
};
|
|
185
185
|
/** Files that the notes pane shows. */
|
|
186
186
|
const NOTE_TYPES = new Set([".md", ".markdown"]);
|
|
187
|
+
/** Images that a note shows (a figure cut out of a PDF, for example). */
|
|
188
|
+
const IMAGE_TYPES = { ".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".gif": "image/gif", ".webp": "image/webp" };
|
|
187
189
|
/** Files that the Open button may hand to the system (never programs or scripts). */
|
|
188
190
|
export const OPENABLE = new Set([".md", ".markdown", ".txt", ".pdf", ".bib", ".csv", ".qmd"]);
|
|
189
191
|
/** A file inside the working folder or the Sub-Sub folder, judged by its real path, so a link cannot point outside. */
|
|
@@ -678,6 +680,19 @@ export async function startWeb(opts, env = process.env, log = () => { }) {
|
|
|
678
680
|
return send(res, 413, { error: "too large to show" });
|
|
679
681
|
return send(res, 200, { path: file, name: basename(file), text: readFileSync(file, "utf8") });
|
|
680
682
|
}
|
|
683
|
+
case "GET /api/note-file": {
|
|
684
|
+
const raw = url.searchParams.get("path") ?? "";
|
|
685
|
+
const file = allowedFile(raw, cwd, cfg.vault) ?? allowedFile(raw, notesRoot, cfg.vault);
|
|
686
|
+
if (!file)
|
|
687
|
+
return send(res, 404, { error: "not found in the Sub-Sub folder" });
|
|
688
|
+
const type = IMAGE_TYPES[extname(file).toLowerCase()];
|
|
689
|
+
if (!type)
|
|
690
|
+
return send(res, 415, { error: "not an image" });
|
|
691
|
+
if (statSync(file).size > 20_000_000)
|
|
692
|
+
return send(res, 413, { error: "too large to show" });
|
|
693
|
+
res.writeHead(200, { ...common, "Content-Type": type, "Cache-Control": "no-store", "X-Content-Type-Options": "nosniff" });
|
|
694
|
+
return void createReadStream(file).pipe(res);
|
|
695
|
+
}
|
|
681
696
|
case "GET /api/library": {
|
|
682
697
|
const q = (url.searchParams.get("q") ?? "").replace(/\s+/g, " ").trim().slice(0, 200);
|
|
683
698
|
if (!q)
|
|
@@ -708,10 +723,12 @@ export async function startWeb(opts, env = process.env, log = () => { }) {
|
|
|
708
723
|
return send(res, 404, { error: "no such session" });
|
|
709
724
|
if (body.path === (await currentSession()))
|
|
710
725
|
return send(res, 409, { error: "the open conversation" });
|
|
711
|
-
// Nothing is deleted: the file moves to a trash folder next to the agent folder
|
|
712
|
-
|
|
713
|
-
|
|
714
|
-
|
|
726
|
+
// Nothing is deleted: the file moves to a trash folder next to the agent folder, in the same
|
|
727
|
+
// subfolder as under sessions/, so that moving it back is one step.
|
|
728
|
+
const rel = relative(join(agentDir, "sessions"), String(body.path));
|
|
729
|
+
const to = join(dirname(agentDir), "trash", rel.startsWith("..") ? basename(rel) : rel);
|
|
730
|
+
mkdirSync(dirname(to), { recursive: true });
|
|
731
|
+
renameSync(String(body.path), to);
|
|
715
732
|
return send(res, 200, { ok: true });
|
|
716
733
|
}
|
|
717
734
|
case "POST /api/open": {
|
package/package.json
CHANGED
package/prompts/compare.md
CHANGED
|
@@ -12,7 +12,7 @@ Make a short slug: lower case, hyphens, at most five words. Use today's date (YY
|
|
|
12
12
|
2. Read each source before you describe it: zotero_get_fulltext for library items, scholar_read_oa_fulltext (methods and results) for open-access works, scholar_get_work for the abstract of the others. Note whether you read the full text or only the abstract.
|
|
13
13
|
3. Build the matrix, one row per source:
|
|
14
14
|
| # | Source | Claim on the question | Evidence type | Population and setting | Caveats | Confidence |
|
|
15
|
-
- Evidence type: for example randomised trial, cohort, case-control, cross-sectional, diagnostic accuracy, systematic review, guideline, expert opinion, modelling.
|
|
15
|
+
- Evidence type: for example randomised trial, cohort, case-control, cross-sectional, diagnostic accuracy, systematic review, guideline, expert opinion, modelling. Add "(preprint)" to a work that is not peer reviewed yet.
|
|
16
16
|
- Caveats: risk of bias, small sample, surrogate outcome, short follow-up, industry funding, abstract only.
|
|
17
17
|
- Confidence: high, moderate or low, with one reason.
|
|
18
18
|
4. Below the matrix, write: where the sources agree; where they disagree and the most likely reason (population, design, outcome definition, analysis); what no source answers. Cite source numbers in every sentence. Write "inference:" before a statement that no single source makes.
|
package/prompts/gaps.md
CHANGED
|
@@ -2,4 +2,4 @@
|
|
|
2
2
|
description: "What am I missing on a topic"
|
|
3
3
|
argument-hint: "<topic or question>"
|
|
4
4
|
---
|
|
5
|
-
Search PubMed and OpenAlex for: $ARGUMENTS. Tell me how many results are already in my library and list the most relevant ones that are missing, one line each. Ask before you import or queue anything.
|
|
5
|
+
Search PubMed and OpenAlex for: $ARGUMENTS. Tell me how many results are already in my library and list the 5 to 15 most relevant ones that are missing, the best fit first, one line each. This is a list to choose from: do not read the works, and do not search again unless the first search found almost nothing. Ask before you import or queue anything.
|
package/prompts/lit.md
CHANGED
|
@@ -12,7 +12,9 @@ Make a short slug from the topic: lower case, hyphens, at most five words. Use t
|
|
|
12
12
|
2. Decide the scale:
|
|
13
13
|
- Narrow question ("what is X", one fact, one test): at most 10 tool calls, a short answer, no plan file.
|
|
14
14
|
- Review question: write the plan to `Research/.plans/<slug>.md` in the Sub-Sub folder: the question, the search phrasings, the study designs that answer it (for example randomised trials, cohorts, diagnostic accuracy studies, systematic reviews), the years, and a task list.
|
|
15
|
-
3.
|
|
15
|
+
3. Rate how hard the search is, from 1 (one named test, drug or condition) to 10 (a broad question across populations, designs and fields). The rating sets the follow-up rounds of section 4: 1 to 3, none; 4 to 7, one; 8 to 10, two. The rounds are a limit, not a target.
|
|
16
|
+
4. Set one publication window from the question (for example 2015 onwards for current practice), or none when the question does not need one. Keep the same window in every search, the follow-up rounds included.
|
|
17
|
+
5. Tell the user the plan in three lines, with the rating and the window. Then continue at once. Do not wait for an answer unless the user asked to see the plan first.
|
|
16
18
|
|
|
17
19
|
## 2. Gather
|
|
18
20
|
|
|
@@ -23,7 +25,7 @@ Make a short slug from the topic: lower case, hyphens, at most five words. Use t
|
|
|
23
25
|
- Library items: zotero_get_fulltext, or the abstract when there is no full text.
|
|
24
26
|
- Open-access works: scholar_read_oa_fulltext, first the section list, then only the sections you need (usually methods and results).
|
|
25
27
|
- Other works: scholar_get_work for the abstract.
|
|
26
|
-
5. Keep an evidence table as you read. Give each source a number that does not change: [1], [2], ... At most two rows from one paper, so that no single paper dominates. In the Read column write exactly one of: full text, abstract, metadata.
|
|
28
|
+
5. Keep an evidence table as you read. Give each source a number that does not change: [1], [2], ... At most two rows from one paper, so that no single paper dominates. In the Read column write exactly one of: full text, abstract, metadata. In the Design column, mark a preprint: for example "cohort (preprint)".
|
|
27
29
|
|
|
28
30
|
## 3. Synthesise
|
|
29
31
|
|
|
@@ -35,9 +37,11 @@ Make a short slug from the topic: lower case, hyphens, at most five words. Use t
|
|
|
35
37
|
|
|
36
38
|
## 4. Check the draft for gaps
|
|
37
39
|
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
40
|
+
Do as many rounds as section 1 allows (none, one or two). Stop early when the sources cover the question.
|
|
41
|
+
|
|
42
|
+
1. Read your draft as a critical reviewer. Name at most three gaps that matter for the question (a population, outcome, design or recent period that the draft does not cover), each as one self-contained search question. A gap is a new angle: do not search again with other words for what the first searches covered.
|
|
43
|
+
2. Search only for those questions (scholar_search_multi, with the window of section 1), read what you add, and give the new sources the next numbers.
|
|
44
|
+
3. Edit only the parts of the draft that a gap names. Do not shorten the draft by more than a tenth. In Methods, give the rating, the rounds you did and what each found (or that it found nothing new).
|
|
41
45
|
|
|
42
46
|
## 5. Verify
|
|
43
47
|
|
|
@@ -86,6 +90,21 @@ reference_check: "<path of the Starbuck report, or none>"
|
|
|
86
90
|
1. [@citekey] for library items; for other works: Authors, year, title, journal. doi:... (not in library)
|
|
87
91
|
```
|
|
88
92
|
|
|
93
|
+
Under Methods, after the text, draw the flow of works with the counts of the screening log, in this form (Obsidian draws it, and so does Sub-Sub's browser view):
|
|
94
|
+
|
|
95
|
+
```mermaid
|
|
96
|
+
flowchart TD
|
|
97
|
+
A["Found by the searches: <n>"] --> B["Unique works: <n>"]
|
|
98
|
+
B -.-> X1["Excluded by the pre-screen: <n>"]
|
|
99
|
+
B --> C["Screened: <n>"]
|
|
100
|
+
C -.-> X2["Excluded at screening: <n>"]
|
|
101
|
+
C --> D["Read: <n>"]
|
|
102
|
+
D -.-> X3["Read, not used: <n>"]
|
|
103
|
+
D --> E["Used in this note: <n>"]
|
|
104
|
+
```
|
|
105
|
+
|
|
106
|
+
The counts must add up: unique works = pre-screen excluded + screened; screened = excluded at screening + read; read = not used + used.
|
|
107
|
+
|
|
89
108
|
If a step fails (a service does not answer, a full text cannot be read), do not stop. Finish the note, mark the step "BLOCKED: <what failed>", and set verification: BLOCKED.
|
|
90
109
|
|
|
91
110
|
## A reading list
|
|
@@ -107,6 +126,6 @@ When the user asks for a reading list (for example `/lit reading list: <topic>`,
|
|
|
107
126
|
## Sources
|
|
108
127
|
```
|
|
109
128
|
|
|
110
|
-
"What it studies" says what the work asks and how, never what it found. Do not state results, effect sizes or conclusions anywhere in the note. Then give the user the path, the first work to read and why, and the works worth adding to the library.
|
|
129
|
+
The Methods section has the same flow diagram, with "On the reading list: <n>" as the last box. "What it studies" says what the work asks and how, never what it found. Do not state results, effect sizes or conclusions anywhere in the note. Then give the user the path, the first work to read and why, and the works worth adding to the library.
|
|
111
130
|
|
|
112
|
-
Before you answer, read the note back to confirm that it exists.
|
|
131
|
+
Before you answer, read the note back to confirm that it exists. For a review, give the user the path, the answer in three lines, and the works worth adding to the library (offer to import them). For a reading list, give the user what the section above says, with no results.
|
package/roles/researcher.md
CHANGED
|
@@ -9,9 +9,11 @@ You read the Zotero library (zotero_ tools), search PubMed, Europe PMC and OpenA
|
|
|
9
9
|
- Every claim about a paper needs a source you actually read: the library (zotero_get_item, zotero_get_fulltext) or an outside record (scholar_get_work). Say which one.
|
|
10
10
|
- Cite library items as [@citekey]. For works not in the library, give DOI or PMID and say that they are not in the library.
|
|
11
11
|
- Do not invent references, numbers or quotes. If you are not sure, say so.
|
|
12
|
+
- Copy quotes word for word, as complete sentences or passages. Mark every omission with [...]; never join two passages into one quote.
|
|
12
13
|
- Read before you summarise. Do not describe a work's methods or results from its title, or from memory. If you read only the abstract, say so.
|
|
13
|
-
- Every literature note (front matter with a citekey) has evidence: full text, abstract or metadata. Give page numbers only when you read the full text. Sub-Sub refuses a note that breaks this rule.
|
|
14
|
+
- Every literature note (front matter with a citekey) has evidence: full text, abstract or metadata. Give page numbers only when you read the full text, as (p. N) right after the quote: Sub-Sub links each one to that page of the PDF in Zotero. Sub-Sub refuses a note that breaks this rule.
|
|
14
15
|
- Mark inferences: write "inference:" before a statement that combines sources and that no single source makes.
|
|
16
|
+
- A preprint (type preprint in the search results, or a venue such as medRxiv, bioRxiv, Research Square, SSRN or arXiv) is not peer reviewed. Say so wherever you use it: "(preprint)" after its design in tables, "preprint, not peer reviewed" in the text. When the same work has a journal version, use and cite the journal version.
|
|
15
17
|
- Never write "verified" or "checked" for a check you did not run.
|
|
16
18
|
- If a step fails (a service does not answer, a full text cannot be read), do not stop: finish the work, and mark the failed step "BLOCKED: <what failed>".
|
|
17
19
|
- Source order: the library first, then PubMed, Europe PMC and OpenAlex. Prefer PubMed and Europe PMC for clinical and physiology questions, OpenAlex for informatics, education and books.
|
|
@@ -35,7 +37,7 @@ You read the Zotero library (zotero_ tools), search PubMed, Europe PMC and OpenA
|
|
|
35
37
|
1. If the question is vague, clarify it in one line.
|
|
36
38
|
2. Search with scholar_search_multi (three or four phrasings, all sources in one call). Use scholar_search_pubmed, scholar_search_europepmc or scholar_search_openalex alone only for a precise query in that service's syntax. Report total hits and how many are already in the library.
|
|
37
39
|
3. Read open-access full text with scholar_read_oa_fulltext: first the section list, then only the sections you need. When a work you need is not open access, say that it may open through {{user}}'s institution (the DOI link, opened on the institution's network), and offer /request-copy to ask the author for a copy. Never point to unlicensed copies (Sci-Hub, LibGen and similar).
|
|
38
|
-
4. List the most relevant works not in the library, one line each on why it matters.
|
|
40
|
+
4. List the most relevant works not in the library, one line each on why it matters. When the user asks for a list (what to read, what is missing), give the ranked list and stop: read a work only when a claim needs it.
|
|
39
41
|
5. Ask which to add. Import the chosen works with zotero_import_identifiers (one preview for all), then offer to tag them. When {{user}} wants to choose later, put them in the import queue with scholar_queue_imports; {{user}} ticks the lines and you import them with zotero_import_queue.
|
|
40
42
|
|
|
41
43
|
## Imports, metadata, PDFs
|
|
@@ -90,6 +92,7 @@ If zotero_get_vocabulary has no topic/ tags, propose topics before tagging: read
|
|
|
90
92
|
|
|
91
93
|
- Follow the formats in the shared rules.
|
|
92
94
|
- After a literature note is written, offer scholar_attach_note with 3 to 4 lines of summary and the path of the note.
|
|
95
|
+
- When the zotero_get_figure tool is available and a figure or table of the paper carries a result that the note discusses, call it with the label of the caption (for example "Figure 2" or "Table 1") and put the Markdown line it returns in the note, with its page link. Quote the caption; do not describe what you cannot see. If it took the whole page (method: page), say so under the image.
|
|
93
96
|
- For a synthesis: show the candidate list first and let {{user}} choose. Read full texts only for the chosen items.
|
|
94
97
|
- For a synthesis of a tag ("what does my library say about X"): find the items with zotero_find_items tags=["topic/x"] and include every one. The table has one row per item; if you leave an item out, say which and why.
|
|
95
98
|
- Say for each item whether you used the full text or the abstract. Never state a number or a claim that is not in what you read.
|
package/src/config.ts
CHANGED
|
@@ -62,7 +62,7 @@ export function uiLanguage(language: string | undefined): "pt" | "en" {
|
|
|
62
62
|
}
|
|
63
63
|
|
|
64
64
|
/** The Zotero server release that this Sub-Sub version runs when there is no local server folder. */
|
|
65
|
-
export const SERVER_VERSION = "0.
|
|
65
|
+
export const SERVER_VERSION = "0.6.0";
|
|
66
66
|
|
|
67
67
|
/** The Starbuck release (PyPI) that this Sub-Sub version runs when there is no local Starbuck folder. */
|
|
68
68
|
export const STARBUCK_VERSION = "0.2.0";
|
package/src/feedback.ts
ADDED
|
@@ -0,0 +1,70 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* /feedback: a draft bug report or idea in the Inbox, for the user to read and post on GitHub.
|
|
3
|
+
* Sub-Sub sends nothing: the user decides what to share. The home and Sub-Sub folders, email
|
|
4
|
+
* addresses and keys are taken out before the draft is written.
|
|
5
|
+
*/
|
|
6
|
+
|
|
7
|
+
export const ISSUES_URL = "https://github.com/tiagojct/subsub/issues/new";
|
|
8
|
+
|
|
9
|
+
export interface FeedbackFacts {
|
|
10
|
+
what: string;
|
|
11
|
+
date: string;
|
|
12
|
+
/** For example "Sub-Sub 0.17.0, pi 1.0.4, Node v24.1.0, darwin arm64". */
|
|
13
|
+
versions: string;
|
|
14
|
+
/** Mode, model, language, add-ons, Zotero. */
|
|
15
|
+
setup: string[];
|
|
16
|
+
/** The output of subsub doctor. */
|
|
17
|
+
checks: string;
|
|
18
|
+
lastError?: string;
|
|
19
|
+
home: string;
|
|
20
|
+
vault?: string;
|
|
21
|
+
}
|
|
22
|
+
|
|
23
|
+
/** Text made safe to post: the Sub-Sub and home folders, email addresses and keys taken out. */
|
|
24
|
+
export function redact(text: string, home: string, vault?: string): string {
|
|
25
|
+
let out = text;
|
|
26
|
+
if (vault) out = out.split(vault).join("<Sub-Sub folder>");
|
|
27
|
+
if (home) out = out.split(home).join("~");
|
|
28
|
+
return out
|
|
29
|
+
.replace(/[\w.+-]+@[\w-]+(?:\.[\w-]+)+/g, "<email>")
|
|
30
|
+
.replace(/\b(?:sk-|AIza|ghp_|gho_|github_pat_|xox[bp]-)[\w-]{8,}/g, "<key>")
|
|
31
|
+
.replace(/\b[A-Za-z0-9_-]{32,}\b/g, "<key>");
|
|
32
|
+
}
|
|
33
|
+
|
|
34
|
+
export function feedbackNote(f: FeedbackFacts): string {
|
|
35
|
+
const clean = (s: string) => redact(s, f.home, f.vault);
|
|
36
|
+
return [
|
|
37
|
+
"---",
|
|
38
|
+
"type: feedback",
|
|
39
|
+
`created: ${f.date}`,
|
|
40
|
+
"---",
|
|
41
|
+
"",
|
|
42
|
+
"# Sub-Sub feedback",
|
|
43
|
+
"",
|
|
44
|
+
`Read this draft, take out anything you do not want to share, then post it at ${ISSUES_URL}. Sub-Sub has sent nothing.`,
|
|
45
|
+
"",
|
|
46
|
+
"## What happened",
|
|
47
|
+
"",
|
|
48
|
+
clean(f.what.trim()) || "(what you did, and what you saw)",
|
|
49
|
+
"",
|
|
50
|
+
"## What you expected",
|
|
51
|
+
"",
|
|
52
|
+
"(what should have happened)",
|
|
53
|
+
"",
|
|
54
|
+
"## Set-up",
|
|
55
|
+
"",
|
|
56
|
+
`- ${clean(f.versions)}`,
|
|
57
|
+
...f.setup.map((l) => `- ${clean(l)}`),
|
|
58
|
+
"",
|
|
59
|
+
"## subsub doctor",
|
|
60
|
+
"",
|
|
61
|
+
"```",
|
|
62
|
+
clean(f.checks.trim()),
|
|
63
|
+
"```",
|
|
64
|
+
"",
|
|
65
|
+
"## Last error",
|
|
66
|
+
"",
|
|
67
|
+
f.lastError ? ["```", clean(f.lastError.trim()).slice(0, 2000), "```"].join("\n") : "(none in this session)",
|
|
68
|
+
"",
|
|
69
|
+
].join("\n");
|
|
70
|
+
}
|
package/src/notes.ts
CHANGED
|
@@ -83,12 +83,17 @@ export function matchable(s: string): string {
|
|
|
83
83
|
.replace(/[^\p{L}\p{N}]+/gu, "");
|
|
84
84
|
}
|
|
85
85
|
|
|
86
|
+
/** The full text as matchable pages joined by "|": Zotero marks a page break with a form feed. */
|
|
87
|
+
export function sourceOf(fullText: string): string {
|
|
88
|
+
return fullText.split("\f").map(matchable).join("|");
|
|
89
|
+
}
|
|
90
|
+
|
|
86
91
|
/**
|
|
87
92
|
* The quotes that are not in the full text. A quote with an ellipsis is checked in parts;
|
|
88
93
|
* square brackets (an editor's insertion) are left out of the comparison.
|
|
89
94
|
*/
|
|
90
95
|
export function missingQuotes(note: string, fullText: string, already: string[] = []): string[] {
|
|
91
|
-
const source =
|
|
96
|
+
const source = sourceOf(fullText);
|
|
92
97
|
const old = new Set(already);
|
|
93
98
|
return quotesIn(note)
|
|
94
99
|
.filter((q) => !old.has(q))
|
|
@@ -99,29 +104,58 @@ export function missingQuotes(note: string, fullText: string, already: string[]
|
|
|
99
104
|
}
|
|
100
105
|
|
|
101
106
|
/**
|
|
102
|
-
*
|
|
103
|
-
*
|
|
107
|
+
* Where a passage starts in the source (from sourceOf), or -1. It must be there word for word.
|
|
108
|
+
* The only gap allowed is a page break with the running footer and header around it (groups of
|
|
109
|
+
* three words in order, at most `gap` characters apart), so that a quote cannot join two passages
|
|
110
|
+
* of one page without marking the omission.
|
|
104
111
|
*/
|
|
105
|
-
export function
|
|
106
|
-
|
|
112
|
+
export function locate(source: string, passage: string, gap = 300): number {
|
|
113
|
+
const whole = source.indexOf(matchable(passage));
|
|
114
|
+
if (whole >= 0) return whole;
|
|
107
115
|
const words = passage.split(/\s+/).map(matchable).filter(Boolean);
|
|
108
116
|
const groups: string[] = [];
|
|
109
117
|
for (let i = 0; i < words.length; i += 3) groups.push(words.slice(i, i + 3).join(""));
|
|
110
|
-
if (groups.length < 2) return
|
|
118
|
+
if (groups.length < 2) return -1;
|
|
111
119
|
for (let start = source.indexOf(groups[0]), tries = 0; start >= 0 && tries < 50; start = source.indexOf(groups[0], start + 1), tries++) {
|
|
112
120
|
let end = start + groups[0].length;
|
|
113
121
|
let ok = true;
|
|
114
122
|
for (const g of groups.slice(1)) {
|
|
115
123
|
const at = source.indexOf(g, end);
|
|
116
|
-
if (at < 0 || at - end > gap) {
|
|
124
|
+
if (at < 0 || (at > end && (at - end > gap || !source.slice(end, at).includes("|")))) {
|
|
117
125
|
ok = false;
|
|
118
126
|
break;
|
|
119
127
|
}
|
|
120
128
|
end = at + g.length;
|
|
121
129
|
}
|
|
122
|
-
if (ok) return
|
|
130
|
+
if (ok) return start;
|
|
123
131
|
}
|
|
124
|
-
return
|
|
132
|
+
return -1;
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
export function appearsIn(source: string, passage: string, gap = 300): boolean {
|
|
136
|
+
return locate(source, passage, gap) >= 0;
|
|
137
|
+
}
|
|
138
|
+
|
|
139
|
+
/** The page of the PDF (1-based, as Zotero's reader counts) where a passage starts, or undefined. */
|
|
140
|
+
export function pageOf(source: string, passage: string): number | undefined {
|
|
141
|
+
const at = locate(source, passage);
|
|
142
|
+
return at < 0 ? undefined : source.slice(0, at).split("|").length;
|
|
143
|
+
}
|
|
144
|
+
|
|
145
|
+
/**
|
|
146
|
+
* Each "(p. N)" after a quote becomes a link that opens the PDF in Zotero at the page where the quote
|
|
147
|
+
* is. The label stays as written (the page printed on the paper may differ from the PDF's count).
|
|
148
|
+
*/
|
|
149
|
+
export function withPageLinks(text: string, source: string, attachment: string): string {
|
|
150
|
+
const link = (quote: string, label: string) => {
|
|
151
|
+
const page = pageOf(source, quote.split(/\s*(?:\.\.\.|…|\[\s*(?:\.\.\.|…)\s*\])\s*/)[0]);
|
|
152
|
+
return page ? `([${label}](zotero://open-pdf/library/items/${attachment}?page=${page}))` : `(${label})`;
|
|
153
|
+
};
|
|
154
|
+
const PAGE = String.raw`\((pp?\.\s?\d+(?:\s?[-–]\s?\d+)?)\)`;
|
|
155
|
+
return text
|
|
156
|
+
.replace(new RegExp(String.raw`(")([^"\n]{25,600})("\s*)${PAGE}`, "g"), (_m, o, q, c, label) => `${o}${q}${c}${link(q, label)}`)
|
|
157
|
+
.replace(new RegExp(String.raw`(“)([^“”\n]{25,600})(”\s*)${PAGE}`, "g"), (_m, o, q, c, label) => `${o}${q}${c}${link(q, label)}`)
|
|
158
|
+
.replace(new RegExp(String.raw`^(>[ \t]*)(.{25,}?)(\s*)${PAGE}[ \t]*$`, "gm"), (_m, o, q, c, label) => `${o}${q}${c}${link(q.replace(/^["“]|["”]$/g, ""), label)}`);
|
|
125
159
|
}
|
|
126
160
|
|
|
127
161
|
export function quoteProblem(missing: string[]): string | undefined {
|
|
@@ -130,6 +164,6 @@ export function quoteProblem(missing: string[]): string | undefined {
|
|
|
130
164
|
return [
|
|
131
165
|
`${missing.length} quote(s) in this note are not in the item's full text:`,
|
|
132
166
|
...shown,
|
|
133
|
-
"Copy each quote word for word from the full text (zotero_get_fulltext)
|
|
167
|
+
"Copy each quote word for word from the full text (zotero_get_fulltext). Mark every omission with [...]: two passages joined without that mark are not a quote. Or remove the quote marks and say it in your own words. Then write the note again.",
|
|
134
168
|
].join("\n");
|
|
135
169
|
}
|
package/src/read.ts
CHANGED
|
@@ -77,6 +77,7 @@ function depth(r: ReadRequest): string[] {
|
|
|
77
77
|
"Depth: the whole paper, section by section. A critique needs the full text: if you can read only the abstract, say so, write what the abstract allows, and offer /request-copy to ask the authors for the paper.",
|
|
78
78
|
'Under "## Methods", "## Results" and "## Discussion": the passages that matter to judge the study, quoted with their place, each with one question for the user (for example: how were the participants selected? what was compared? what else could explain the result?).',
|
|
79
79
|
`Under "## Appraisal": name the design and the reporting guideline that applies (${GUIDELINES}), and tell the user to check the paper against it item by item. Do not copy the checklist. Then five to eight questions on the risk of bias for this design (selection, confounding, measurement, missing data, selective reporting), each with the passage to look at.`,
|
|
80
|
+
"If the zotero_get_figure tool is available, show the table or figure with the main result: call it with the label of its caption and put the Markdown line it returns under the passage that cites it, with the page link, and a question about what the user sees in it.",
|
|
80
81
|
];
|
|
81
82
|
}
|
|
82
83
|
}
|
|
@@ -98,7 +99,7 @@ export function readInstruction(r: ReadRequest): string {
|
|
|
98
99
|
"",
|
|
99
100
|
"Rules:",
|
|
100
101
|
"- State no results, effect sizes, summaries or conclusions, in the note or in the chat. Quote the paper and ask; the user writes the answers.",
|
|
101
|
-
"- Copy quotes word for word from what you read. After each quote give its place:
|
|
102
|
+
"- Copy quotes word for word from what you read, and mark every omission with [...]. After each quote give its place: (p. N) only with evidence: full text (Sub-Sub links it to that page of the PDF), otherwise the section (Abstract, Methods, Results).",
|
|
102
103
|
`- The note is Literature/<citekey>.md in the Sub-Sub folder, with the front matter of a literature note from the shared rules plus type: reading-note and purpose: ${r.purpose}; evidence says what you read. If the note exists, do not overwrite it: add a section "## Reading ${r.date}: ${r.purpose}" at the end with edit, with the content below one heading level lower.`,
|
|
103
104
|
'- Start with "## Why I am reading this" (the reason, in the user\'s words when they gave them) and end with one line "Next step:" (what to read or do next).',
|
|
104
105
|
"",
|
package/src/subsub.ts
CHANGED
|
@@ -16,6 +16,7 @@
|
|
|
16
16
|
|
|
17
17
|
import type { ExtensionAPI, ExtensionContext } from "@earendil-works/pi-coding-agent";
|
|
18
18
|
import { existsSync, mkdirSync, readdirSync, readFileSync, writeFileSync } from "node:fs";
|
|
19
|
+
import { homedir } from "node:os";
|
|
19
20
|
import { dirname, join } from "node:path";
|
|
20
21
|
import { fileURLToPath } from "node:url";
|
|
21
22
|
import { Bridge, type ServerSpec } from "./bridge.ts";
|
|
@@ -35,8 +36,10 @@ import {
|
|
|
35
36
|
} from "./config.ts";
|
|
36
37
|
import { coerceArgs, loosenArrays, resolvePathArgs } from "./args.ts";
|
|
37
38
|
import { aiStatement } from "./statement.ts";
|
|
39
|
+
import { feedbackNote, ISSUES_URL } from "./feedback.ts";
|
|
40
|
+
import { formatChecks, runChecks } from "./doctor.ts";
|
|
38
41
|
import { PURPOSES, type Purpose, readArgs, readInstruction, type ReadRequest } from "./read.ts";
|
|
39
|
-
import { applyEdits, checkLiteratureNote, field, frontMatter, itemKeyOf, missingQuotes, quoteProblem, quotesIn } from "./notes.ts";
|
|
42
|
+
import { applyEdits, checkLiteratureNote, field, frontMatter, itemKeyOf, missingQuotes, quoteProblem, quotesIn, sourceOf, withPageLinks } from "./notes.ts";
|
|
40
43
|
import { buildPolicy, isSecret, judgePath, normalizeToolPath, realish, within } from "./paths.ts";
|
|
41
44
|
import { describeArgs, formatPreview, isEmptyPreview } from "./preview.ts";
|
|
42
45
|
import { perMinuteWait, providerTrouble, TROUBLE_TEXT, TROUBLE_TEXT_PT } from "./keys.ts";
|
|
@@ -52,6 +55,8 @@ export interface SubsubDeps {
|
|
|
52
55
|
/** A ready bridge (tests). When given, Sub-Sub does not start or stop servers. */
|
|
53
56
|
bridge?: Bridge;
|
|
54
57
|
specs?: ServerSpec[];
|
|
58
|
+
/** The subsub doctor lines for /feedback (tests give their own). */
|
|
59
|
+
doctor?: () => string;
|
|
55
60
|
}
|
|
56
61
|
|
|
57
62
|
const PACKAGE_DIR = join(dirname(fileURLToPath(import.meta.url)), "..");
|
|
@@ -154,6 +159,8 @@ export async function createSubsub(pi: ExtensionAPI, deps: SubsubDeps = {}): Pro
|
|
|
154
159
|
/** True under `subsub web`: pi runs in RPC mode and the web view shows the state that the terminal header shows. */
|
|
155
160
|
const webView = process.env.SUBSUB_WEB === "1";
|
|
156
161
|
let lastError: string | undefined;
|
|
162
|
+
/** The last provider or tool error of the session, for /feedback. */
|
|
163
|
+
let recentError: string | undefined;
|
|
157
164
|
let resumes = 0;
|
|
158
165
|
let resumeTimer: ReturnType<typeof setTimeout> | undefined;
|
|
159
166
|
let turnStart = 0;
|
|
@@ -355,12 +362,18 @@ export async function createSubsub(pi: ExtensionAPI, deps: SubsubDeps = {}): Pro
|
|
|
355
362
|
pi.on("message_end", async (event, ctx) => {
|
|
356
363
|
const m = event.message as { role?: string; stopReason?: string; errorMessage?: string };
|
|
357
364
|
if (m.role !== "assistant" || m.stopReason !== "error") return;
|
|
358
|
-
lastError = m.errorMessage;
|
|
365
|
+
lastError = recentError = m.errorMessage;
|
|
359
366
|
if (webView || perMinuteWait(m.errorMessage) !== undefined) return;
|
|
360
367
|
const kind = providerTrouble(m.errorMessage);
|
|
361
368
|
if (kind) ctx.ui.notify(`Sub-Sub: ${trouble[kind]}`, "warning");
|
|
362
369
|
});
|
|
363
370
|
|
|
371
|
+
pi.on("tool_result", async (event) => {
|
|
372
|
+
if (!event.isError) return;
|
|
373
|
+
const text = event.content.map((c) => (c.type === "text" ? c.text : "")).join(" ");
|
|
374
|
+
recentError = `${(event as { toolName?: string }).toolName ?? "tool"}: ${text}`.slice(0, 2000);
|
|
375
|
+
});
|
|
376
|
+
|
|
364
377
|
// A per-minute limit (free tiers): wait as long as the provider says, then continue the task,
|
|
365
378
|
// up to three times per request of the user. Anything the user types cancels the wait.
|
|
366
379
|
pi.on("agent_end", async (_event, ctx) => {
|
|
@@ -527,28 +540,39 @@ export async function createSubsub(pi: ExtensionAPI, deps: SubsubDeps = {}): Pro
|
|
|
527
540
|
if (!front || !key || field(front, "evidence")?.toLowerCase() !== "full text" || !quotesIn(text).length) return undefined;
|
|
528
541
|
const full = await fullTextOf(key, signal);
|
|
529
542
|
// Without the full text (no PDF indexed, Zotero closed) there is nothing to check against.
|
|
530
|
-
|
|
543
|
+
if (!full) return undefined;
|
|
544
|
+
const missing = quoteProblem(missingQuotes(text, full.text, quotesIn(before)));
|
|
545
|
+
// A note that passes is written with its page references linked to the PDF.
|
|
546
|
+
if (!missing && full.attachment) {
|
|
547
|
+
const link = (s: string) => withPageLinks(s, sourceOf(full.text), full.attachment!);
|
|
548
|
+
if (name === "write") input.content = link(String(input.content ?? ""));
|
|
549
|
+
else for (const e of input.edits as Array<{ newText?: unknown }>) if (typeof e.newText === "string") e.newText = link(e.newText);
|
|
550
|
+
}
|
|
551
|
+
return missing;
|
|
531
552
|
}
|
|
532
553
|
|
|
533
|
-
/** The whole indexed full text of an item, read in parts, kept for the session. */
|
|
534
|
-
const fullTexts = new Map<string, string | null>();
|
|
535
|
-
async function fullTextOf(key: string, signal?: AbortSignal): Promise<string | undefined> {
|
|
554
|
+
/** The whole indexed full text of an item and its attachment, read in parts, kept for the session. */
|
|
555
|
+
const fullTexts = new Map<string, { text: string; attachment?: string } | null>();
|
|
556
|
+
async function fullTextOf(key: string, signal?: AbortSignal): Promise<{ text: string; attachment?: string } | undefined> {
|
|
536
557
|
if (fullTexts.has(key)) return fullTexts.get(key) ?? undefined;
|
|
537
558
|
let text = "";
|
|
559
|
+
let attachment: string | undefined;
|
|
538
560
|
let offset: number | null = 0;
|
|
539
561
|
try {
|
|
540
562
|
for (let part = 0; offset !== null && part < 20; part++) {
|
|
541
563
|
const res = await current().call("zotero_get_fulltext", { key, offset, max_chars: 60000 }, signal);
|
|
542
564
|
if (res.isError) break;
|
|
543
|
-
const data = (res.data ?? {}) as { text?: string; next_offset?: number | null };
|
|
565
|
+
const data = (res.data ?? {}) as { text?: string; next_offset?: number | null; attachment?: string };
|
|
544
566
|
text += data.text ?? "";
|
|
567
|
+
attachment ??= data.attachment;
|
|
545
568
|
offset = data.next_offset ?? null;
|
|
546
569
|
}
|
|
547
570
|
} catch {
|
|
548
571
|
/* no full text: the check is skipped */
|
|
549
572
|
}
|
|
550
|
-
|
|
551
|
-
|
|
573
|
+
const full = text ? { text, attachment } : null;
|
|
574
|
+
fullTexts.set(key, full);
|
|
575
|
+
return full ?? undefined;
|
|
552
576
|
}
|
|
553
577
|
|
|
554
578
|
/**
|
|
@@ -744,6 +768,56 @@ export async function createSubsub(pi: ExtensionAPI, deps: SubsubDeps = {}): Pro
|
|
|
744
768
|
},
|
|
745
769
|
});
|
|
746
770
|
|
|
771
|
+
pi.registerCommand("feedback", {
|
|
772
|
+
description: "Sub-Sub: a draft bug report or idea in Inbox/, to read and post on GitHub yourself; Sub-Sub sends nothing [what happened]",
|
|
773
|
+
handler: async (args, ctx) => {
|
|
774
|
+
const date = new Date().toISOString().slice(0, 10);
|
|
775
|
+
let piVersion = "";
|
|
776
|
+
try {
|
|
777
|
+
piVersion = JSON.parse(readFileSync(join(dirname(fileURLToPath(import.meta.resolve("@earendil-works/pi-coding-agent"))), "..", "package.json"), "utf8")).version;
|
|
778
|
+
} catch {
|
|
779
|
+
/* not needed for the report */
|
|
780
|
+
}
|
|
781
|
+
let checks: string;
|
|
782
|
+
try {
|
|
783
|
+
checks = deps.doctor ? deps.doctor() : formatChecks(runChecks(cfg, process.env));
|
|
784
|
+
} catch (err) {
|
|
785
|
+
checks = `subsub doctor failed: ${(err as Error).message}`;
|
|
786
|
+
}
|
|
787
|
+
const text = feedbackNote({
|
|
788
|
+
what: String(args ?? ""),
|
|
789
|
+
date,
|
|
790
|
+
versions: `Sub-Sub ${ownVersion()}${piVersion ? `, pi ${piVersion}` : ""}, Node ${process.version}, ${process.platform} ${process.arch}`,
|
|
791
|
+
setup: [
|
|
792
|
+
`Mode: ${mode}. Model: ${ctx.model ? `${ctx.model.provider}/${ctx.model.id}` : "none"}.`,
|
|
793
|
+
`Language: ${cfg.language}. Reference checks (Starbuck): ${starbuckEnabled(cfg) ? "on" : "off"}.`,
|
|
794
|
+
`Zotero: ${library.zotero}.${bridge && Object.keys(bridge.errors).length ? ` Servers not running: ${Object.keys(bridge.errors).join(", ")}.` : ""}`,
|
|
795
|
+
],
|
|
796
|
+
checks,
|
|
797
|
+
lastError: recentError,
|
|
798
|
+
home: homedir(),
|
|
799
|
+
vault: cfg.vault,
|
|
800
|
+
});
|
|
801
|
+
const dir = join(cfg.vault ?? ctx.cwd, "Inbox");
|
|
802
|
+
let file = join(dir, `${date} Sub-Sub feedback.md`);
|
|
803
|
+
for (let n = 2; existsSync(file); n++) file = join(dir, `${date} Sub-Sub feedback-${n}.md`);
|
|
804
|
+
try {
|
|
805
|
+
mkdirSync(dir, { recursive: true });
|
|
806
|
+
writeFileSync(file, text);
|
|
807
|
+
} catch (err) {
|
|
808
|
+
ctx.ui.notify(`${L("Could not save it", "Não foi possível guardar")}: ${(err as Error).message}`, "warning");
|
|
809
|
+
return;
|
|
810
|
+
}
|
|
811
|
+
ctx.ui.notify(
|
|
812
|
+
L(
|
|
813
|
+
`Sub-Sub: a draft report is in ${file}. Read it, add what you expected, take out anything you do not want to share, and post it at ${ISSUES_URL}. Sub-Sub has sent nothing.`,
|
|
814
|
+
`Sub-Sub: há um rascunho do relatório em ${file}. Leia-o, acrescente o que esperava, retire o que não quer partilhar e publique-o em ${ISSUES_URL}. O Sub-Sub não enviou nada.`,
|
|
815
|
+
),
|
|
816
|
+
"info",
|
|
817
|
+
);
|
|
818
|
+
},
|
|
819
|
+
});
|
|
820
|
+
|
|
747
821
|
pi.registerCommand("history", {
|
|
748
822
|
description: "Sub-Sub: recent library changes",
|
|
749
823
|
handler: async (_args, ctx) => {
|
package/src/web.ts
CHANGED
|
@@ -19,7 +19,7 @@ import { randomBytes, timingSafeEqual } from "node:crypto";
|
|
|
19
19
|
import { createReadStream, existsSync, mkdirSync, readdirSync, readFileSync, renameSync, statSync, writeFileSync, appendFileSync } from "node:fs";
|
|
20
20
|
import { createServer, type IncomingMessage, type Server, type ServerResponse } from "node:http";
|
|
21
21
|
import { homedir } from "node:os";
|
|
22
|
-
import { basename, dirname, extname, join, normalize, resolve, sep } from "node:path";
|
|
22
|
+
import { basename, dirname, extname, join, normalize, relative, resolve, sep } from "node:path";
|
|
23
23
|
import { StringDecoder } from "node:string_decoder";
|
|
24
24
|
import { agentDirFor, chooseCwd, PACKAGE_DIR, packageVersion } from "./cli.ts";
|
|
25
25
|
import { configPath, expand, loadConfig, RECOMMENDED_MODELS, readConfigFile, saveConfig, type SubsubConfig, uiLanguage, withStarbuck } from "./config.ts";
|
|
@@ -203,6 +203,9 @@ const TYPES: Record<string, string> = {
|
|
|
203
203
|
/** Files that the notes pane shows. */
|
|
204
204
|
const NOTE_TYPES = new Set([".md", ".markdown"]);
|
|
205
205
|
|
|
206
|
+
/** Images that a note shows (a figure cut out of a PDF, for example). */
|
|
207
|
+
const IMAGE_TYPES: Record<string, string> = { ".png": "image/png", ".jpg": "image/jpeg", ".jpeg": "image/jpeg", ".gif": "image/gif", ".webp": "image/webp" };
|
|
208
|
+
|
|
206
209
|
/** Files that the Open button may hand to the system (never programs or scripts). */
|
|
207
210
|
export const OPENABLE = new Set([".md", ".markdown", ".txt", ".pdf", ".bib", ".csv", ".qmd"]);
|
|
208
211
|
|
|
@@ -686,6 +689,16 @@ export async function startWeb(opts: WebOptions, env: NodeJS.ProcessEnv = proces
|
|
|
686
689
|
if (statSync(file).size > 2_000_000) return send(res, 413, { error: "too large to show" });
|
|
687
690
|
return send(res, 200, { path: file, name: basename(file), text: readFileSync(file, "utf8") });
|
|
688
691
|
}
|
|
692
|
+
case "GET /api/note-file": {
|
|
693
|
+
const raw = url.searchParams.get("path") ?? "";
|
|
694
|
+
const file = allowedFile(raw, cwd, cfg.vault) ?? allowedFile(raw, notesRoot, cfg.vault);
|
|
695
|
+
if (!file) return send(res, 404, { error: "not found in the Sub-Sub folder" });
|
|
696
|
+
const type = IMAGE_TYPES[extname(file).toLowerCase()];
|
|
697
|
+
if (!type) return send(res, 415, { error: "not an image" });
|
|
698
|
+
if (statSync(file).size > 20_000_000) return send(res, 413, { error: "too large to show" });
|
|
699
|
+
res.writeHead(200, { ...common, "Content-Type": type, "Cache-Control": "no-store", "X-Content-Type-Options": "nosniff" });
|
|
700
|
+
return void createReadStream(file).pipe(res);
|
|
701
|
+
}
|
|
689
702
|
case "GET /api/library": {
|
|
690
703
|
const q = (url.searchParams.get("q") ?? "").replace(/\s+/g, " ").trim().slice(0, 200);
|
|
691
704
|
if (!q) return send(res, 200, { q, items: [] });
|
|
@@ -710,10 +723,12 @@ export async function startWeb(opts: WebOptions, env: NodeJS.ProcessEnv = proces
|
|
|
710
723
|
const known = (await sessions()) as Array<{ path: string }>;
|
|
711
724
|
if (!known.some((x) => x.path === body.path)) return send(res, 404, { error: "no such session" });
|
|
712
725
|
if (body.path === (await currentSession())) return send(res, 409, { error: "the open conversation" });
|
|
713
|
-
// Nothing is deleted: the file moves to a trash folder next to the agent folder
|
|
714
|
-
|
|
715
|
-
|
|
716
|
-
|
|
726
|
+
// Nothing is deleted: the file moves to a trash folder next to the agent folder, in the same
|
|
727
|
+
// subfolder as under sessions/, so that moving it back is one step.
|
|
728
|
+
const rel = relative(join(agentDir, "sessions"), String(body.path));
|
|
729
|
+
const to = join(dirname(agentDir), "trash", rel.startsWith("..") ? basename(rel) : rel);
|
|
730
|
+
mkdirSync(dirname(to), { recursive: true });
|
|
731
|
+
renameSync(String(body.path), to);
|
|
717
732
|
return send(res, 200, { ok: true });
|
|
718
733
|
}
|
|
719
734
|
case "POST /api/open": {
|
|
@@ -41,7 +41,7 @@ Shared rules for Sub-Sub. Sub-Sub reads this file at the start of every session.
|
|
|
41
41
|
- Create a note only when the user asks. Ask before creating more than 10 notes.
|
|
42
42
|
- Write from the full text (get_fulltext) when it exists. Otherwise use the abstract and the Zotero notes, and write "(from abstract)" after the summary.
|
|
43
43
|
- Front matter field evidence: full text, abstract or metadata (what you read). Page numbers only with evidence: full text. Sub-Sub refuses a literature note without this field.
|
|
44
|
-
- Do not copy the abstract in full. Give page
|
|
44
|
+
- Do not copy the abstract in full. Give the page of a quote as (p. N) right after it, when you read the full text; Sub-Sub links it to that page of the PDF in Zotero.
|
|
45
45
|
- If a note for the citekey exists, do not overwrite it. Add a dated section or ask.
|
|
46
46
|
- After the note is written, offer to link it in Zotero with attach_note.
|
|
47
47
|
- /read writes a reading note instead: quotes with their place and questions, which the user answers.
|
package/web/app.css
CHANGED
|
@@ -562,6 +562,7 @@ body.layout-full { --measure: 72rem; }
|
|
|
562
562
|
.msg-action-btn { font-size: 0.75rem; padding: 0.15rem 0.45rem; color: var(--faint); }
|
|
563
563
|
.msg-action-btn:hover { color: var(--text); background: var(--surface); }
|
|
564
564
|
.msg-action-btn.copied { color: var(--add); }
|
|
565
|
+
.table-actions { display: flex; justify-content: flex-end; margin: -0.35rem 0 0.9rem; }
|
|
565
566
|
|
|
566
567
|
/* Tool calls: one quiet line each; the result opens below. */
|
|
567
568
|
.tool { margin: 0.35rem 0; font-size: 0.82rem; color: var(--muted); }
|
|
@@ -856,6 +857,15 @@ body > p.empty { text-align: center; color: var(--muted); padding: 30vh 1rem 0;
|
|
|
856
857
|
.settings button { width: 100%; text-align: left; color: var(--text); }
|
|
857
858
|
.settings .model-btn { font-family: var(--mono); font-size: 0.78rem; overflow: hidden; text-overflow: ellipsis; white-space: nowrap; }
|
|
858
859
|
|
|
860
|
+
/* ---------------------------------------------------------------- the flow of works (/lit) */
|
|
861
|
+
|
|
862
|
+
.flow { display: grid; gap: 1rem; margin: 1rem 0; font-family: var(--sans); }
|
|
863
|
+
.flow-row { display: grid; grid-template-columns: minmax(0, 15rem) repeat(auto-fit, minmax(0, 13rem)); gap: 1.4rem; align-items: center; }
|
|
864
|
+
.flow-box { position: relative; padding: 0.45rem 0.7rem; border: 1px solid var(--rule-strong); border-radius: var(--radius); background: var(--raised); font-size: 0.84rem; text-align: center; }
|
|
865
|
+
.flow-row + .flow-row .flow-box::before { content: ""; position: absolute; left: 50%; top: calc(-1rem - 1px); height: 1rem; border-left: 1px solid var(--muted); }
|
|
866
|
+
.flow-side { position: relative; padding: 0.35rem 0.6rem; border: 1px dashed var(--rule-strong); border-radius: var(--radius); color: var(--muted); font-size: 0.78rem; }
|
|
867
|
+
.flow-side::before { content: ""; position: absolute; right: 100%; top: 50%; width: 1.4rem; border-top: 1px dashed var(--muted); }
|
|
868
|
+
|
|
859
869
|
/* ---------------------------------------------------------------- library search and notes */
|
|
860
870
|
|
|
861
871
|
#notes-btn { font-family: var(--sans); }
|
|
@@ -903,6 +913,7 @@ body > p.empty { text-align: center; color: var(--muted); padding: 30vh 1rem 0;
|
|
|
903
913
|
.note-list button:hover { background: var(--bg); }
|
|
904
914
|
.note-list .rel { font-size: 0.84rem; overflow-wrap: anywhere; }
|
|
905
915
|
.note-list .when { font-size: 0.72rem; color: var(--faint); }
|
|
916
|
+
.note-body img.note-img { display: block; max-width: 100%; height: auto; margin: 0.8rem 0; border: 1px solid var(--rule); border-radius: var(--radius); background: #fff; }
|
|
906
917
|
a[data-note] { color: var(--accent); text-decoration: underline; text-decoration-color: var(--accent-soft); text-underline-offset: 0.18em; cursor: pointer; }
|
|
907
918
|
|
|
908
919
|
@media (max-width: 64rem) {
|
package/web/app.js
CHANGED
|
@@ -83,11 +83,21 @@ marked.use({
|
|
|
83
83
|
if (!SAFE_LINK.test(token.href ?? "")) return text;
|
|
84
84
|
return `<a href="${escapeHtml(token.href)}" target="_blank" rel="noopener noreferrer">${text}</a>`;
|
|
85
85
|
},
|
|
86
|
+
code(token) {
|
|
87
|
+
return (token.lang === "mermaid" && flowHtml(token.text)) || false;
|
|
88
|
+
},
|
|
86
89
|
codespan(token) {
|
|
87
90
|
const code = `<code>${escapeHtml(token.text)}</code>`;
|
|
88
91
|
return NOTE_PATH.test(token.text) ? noteLink(token.text, code) : code;
|
|
89
92
|
},
|
|
90
93
|
image(token) {
|
|
94
|
+
// In the notes pane, an image next to the note (a figure cut out of the PDF) is shown;
|
|
95
|
+
// anywhere else, and for any other address, only its description.
|
|
96
|
+
const href = decodeURI(token.href ?? "");
|
|
97
|
+
if (noteDir && /^(?![a-z][\w+.-]*:|[\\/])[^?#]+\.(png|jpe?g|gif|webp)$/i.test(href)) {
|
|
98
|
+
const src = `/api/note-file?path=${encodeURIComponent(`${noteDir}/${href}`)}`;
|
|
99
|
+
return `<img class="note-img" src="${escapeHtml(src)}" alt="${escapeHtml(token.text ?? "")}" loading="lazy">`;
|
|
100
|
+
}
|
|
91
101
|
return escapeHtml(token.text || token.href || "");
|
|
92
102
|
},
|
|
93
103
|
table(token) {
|
|
@@ -110,6 +120,45 @@ marked.use({
|
|
|
110
120
|
},
|
|
111
121
|
},
|
|
112
122
|
});
|
|
123
|
+
/**
|
|
124
|
+
* The flow of works that /lit writes as Mermaid (a chain of boxes, with dotted branches to the side
|
|
125
|
+
* for what was left out), drawn as boxes. Any other diagram stays as code; Obsidian draws it.
|
|
126
|
+
*/
|
|
127
|
+
function flowHtml(src) {
|
|
128
|
+
const lines = String(src).split("\n").map((l) => l.trim()).filter(Boolean);
|
|
129
|
+
if (!/^(flowchart|graph)\s+(TD|TB)\b/.test(lines.shift() ?? "")) return null;
|
|
130
|
+
const label = {};
|
|
131
|
+
const next = {};
|
|
132
|
+
const side = {};
|
|
133
|
+
const into = new Set();
|
|
134
|
+
for (const line of lines) {
|
|
135
|
+
const n = /^(\w+)\["([^"]*)"\]$/.exec(line);
|
|
136
|
+
if (n) {
|
|
137
|
+
label[n[1]] = n[2];
|
|
138
|
+
continue;
|
|
139
|
+
}
|
|
140
|
+
const m = /^(\w+)(?:\["([^"]*)"\])?\s*(-->|-\.->)\s*(\w+)(?:\["([^"]*)"\])?$/.exec(line);
|
|
141
|
+
if (!m) return null;
|
|
142
|
+
const [, a, la, arrow, b, lb] = m;
|
|
143
|
+
if (la !== undefined) label[a] = la;
|
|
144
|
+
if (lb !== undefined) label[b] = lb;
|
|
145
|
+
if (arrow === "-->") {
|
|
146
|
+
if (next[a]) return null; // a branch of the main chain: not this kind of diagram
|
|
147
|
+
next[a] = b;
|
|
148
|
+
into.add(b);
|
|
149
|
+
} else (side[a] ??= []).push(b);
|
|
150
|
+
}
|
|
151
|
+
const rows = [];
|
|
152
|
+
const seen = new Set();
|
|
153
|
+
for (let id = Object.keys(next).find((k) => !into.has(k)); id && !seen.has(id); id = next[id]) {
|
|
154
|
+
seen.add(id);
|
|
155
|
+
const sides = (side[id] ?? []).map((s) => `<div class="flow-side">${escapeHtml(label[s] ?? s)}</div>`).join("");
|
|
156
|
+
rows.push(`<div class="flow-row"><div class="flow-box">${escapeHtml(label[id] ?? id)}</div>${sides}</div>`);
|
|
157
|
+
}
|
|
158
|
+
if (rows.length < 2) return null;
|
|
159
|
+
return `<div class="flow">${rows.join("")}</div>`;
|
|
160
|
+
}
|
|
161
|
+
|
|
113
162
|
function md(text) {
|
|
114
163
|
try {
|
|
115
164
|
return marked.parse(String(text ?? ""));
|
|
@@ -344,6 +393,7 @@ function searchLibrary() {
|
|
|
344
393
|
// ---------------------------------------------------------------- notes pane
|
|
345
394
|
|
|
346
395
|
let noteShown = null; // the note on screen, or null for the list
|
|
396
|
+
let noteDir = ""; // while a note is rendered: its folder, for the images next to it
|
|
347
397
|
let notesList = [];
|
|
348
398
|
|
|
349
399
|
function showPane(open) {
|
|
@@ -406,8 +456,11 @@ async function openNote(path) {
|
|
|
406
456
|
$("note-filter").hidden = true;
|
|
407
457
|
const front = /^---\r?\n([\s\S]*?)\r?\n---(?:\r?\n|$)/.exec(data.text);
|
|
408
458
|
const content = el("div", { class: "md" });
|
|
459
|
+
noteDir = data.path.replace(/[\\/][^\\/]*$/, "");
|
|
409
460
|
content.innerHTML = md(front ? data.text.slice(front[0].length) : data.text);
|
|
461
|
+
noteDir = "";
|
|
410
462
|
attachCopyButtons(content);
|
|
463
|
+
attachTableButtons(content, data.name.replace(/\.md$/i, ""));
|
|
411
464
|
$("note-body").replaceChildren(front ? el("details", { class: "front" }, el("summary", { text: t("properties") }), el("pre", { text: front[1] })) : "", content);
|
|
412
465
|
$("note-body").scrollTop = 0;
|
|
413
466
|
showPane(true);
|
|
@@ -519,17 +572,40 @@ function exportConversation() {
|
|
|
519
572
|
if (!msgs.length) return;
|
|
520
573
|
const date = new Date().toISOString().slice(0, 10);
|
|
521
574
|
const content = `# ${t("exportHeading")} (${date})\n\n${t("exportMode")}: ${t(mode())}\n\n---\n\n${msgs.join("\n---\n\n")}`;
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
525
|
-
|
|
526
|
-
|
|
575
|
+
download(content, `Sub-Sub-${date}.md`, "text/markdown;charset=utf-8");
|
|
576
|
+
}
|
|
577
|
+
|
|
578
|
+
/** Save text as a file in the downloads folder. */
|
|
579
|
+
function download(content, name, type) {
|
|
580
|
+
const url = URL.createObjectURL(new Blob([content], { type }));
|
|
581
|
+
const a = el("a", { href: url, download: name });
|
|
527
582
|
document.body.append(a);
|
|
528
583
|
a.click();
|
|
529
584
|
a.remove();
|
|
530
585
|
URL.revokeObjectURL(url);
|
|
531
586
|
}
|
|
532
587
|
|
|
588
|
+
/** A table as CSV for Excel, Rayyan or Covidence. Excel in Portuguese expects semicolons; the BOM keeps the accents. */
|
|
589
|
+
function tableCsv(table) {
|
|
590
|
+
const sep = lang === "pt" ? ";" : ",";
|
|
591
|
+
const cell = (c) => {
|
|
592
|
+
const v = c.textContent.replace(/\s+/g, " ").trim(); // not innerText: the header is upper case only in CSS
|
|
593
|
+
return /[",;\n]/.test(v) ? `"${v.replaceAll('"', '""')}"` : v;
|
|
594
|
+
};
|
|
595
|
+
return `\ufeff${[...table.rows].map((r) => [...r.cells].map(cell).join(sep)).join("\r\n")}\r\n`;
|
|
596
|
+
}
|
|
597
|
+
|
|
598
|
+
/** A "Download CSV" button under each table (evidence tables, extraction sheets, matrices). */
|
|
599
|
+
function attachTableButtons(root, name) {
|
|
600
|
+
const wraps = [...root.querySelectorAll(".md-table-wrap")];
|
|
601
|
+
wraps.forEach((wrap, i) => {
|
|
602
|
+
if (wrap.nextElementSibling?.classList.contains("table-actions")) return;
|
|
603
|
+
const file = `${name}${wraps.length > 1 ? `-${i + 1}` : ""}.csv`;
|
|
604
|
+
const btn = el("button", { type: "button", class: "msg-action-btn", text: t("downloadCsv"), onclick: () => download(tableCsv(wrap.querySelector("table")), file, "text/csv;charset=utf-8") });
|
|
605
|
+
wrap.after(el("div", { class: "table-actions" }, btn));
|
|
606
|
+
});
|
|
607
|
+
}
|
|
608
|
+
|
|
533
609
|
function closeSide() {
|
|
534
610
|
$("side").classList.remove("open");
|
|
535
611
|
$("menu-btn").setAttribute("aria-expanded", "false");
|
|
@@ -801,6 +877,7 @@ function renderAssistantFinal(message, msg = startAssistant()) {
|
|
|
801
877
|
const node = el("div", { class: "md" });
|
|
802
878
|
node.innerHTML = md(c.text);
|
|
803
879
|
attachCopyButtons(node);
|
|
880
|
+
attachTableButtons(node, `Sub-Sub-${new Date().toISOString().slice(0, 10)}`);
|
|
804
881
|
msg.box.append(node);
|
|
805
882
|
} else if (c.type === "toolCall") {
|
|
806
883
|
const row = toolRow(c.id, c.name, c.arguments);
|
package/web/i18n.js
CHANGED
|
@@ -109,6 +109,7 @@ export const STRINGS = {
|
|
|
109
109
|
notesHint: "The notes in your Sub-Sub folder",
|
|
110
110
|
allNotes: "All notes",
|
|
111
111
|
properties: "Properties",
|
|
112
|
+
downloadCsv: "Download CSV",
|
|
112
113
|
filterNotes: "Filter notes",
|
|
113
114
|
noNotes: "No notes yet.",
|
|
114
115
|
openInApp: "Open in app",
|
|
@@ -248,6 +249,7 @@ export const STRINGS = {
|
|
|
248
249
|
notesHint: "As notas da sua pasta do Sub-Sub",
|
|
249
250
|
allNotes: "Todas as notas",
|
|
250
251
|
properties: "Propriedades",
|
|
252
|
+
downloadCsv: "Transferir CSV",
|
|
251
253
|
filterNotes: "Filtrar notas",
|
|
252
254
|
noNotes: "Ainda não há notas.",
|
|
253
255
|
openInApp: "Abrir na aplicação",
|
|
@@ -293,7 +295,7 @@ export const ACTIONS = {
|
|
|
293
295
|
{ cmd: "lit", en: "Literature review", pt: "Revisão da literatura" },
|
|
294
296
|
{ cmd: "lit reading list:", en: "Reading list", pt: "Lista de leitura" },
|
|
295
297
|
{ cmd: "compare", en: "Compare sources", pt: "Comparar fontes" },
|
|
296
|
-
{ cmd: "lit-note", item: true, en: "Literature note", pt: "Nota de
|
|
298
|
+
{ cmd: "lit-note", item: true, en: "Literature note", pt: "Nota de literatura" },
|
|
297
299
|
{ cmd: "synthesis", en: "Synthesis", pt: "Síntese" },
|
|
298
300
|
{ cmd: "gaps", en: "What is missing", pt: "O que falta" },
|
|
299
301
|
{ cmd: "manuscript", en: "Check a manuscript", pt: "Verificar um manuscrito" },
|
|
@@ -311,6 +313,7 @@ export const ACTIONS = {
|
|
|
311
313
|
{ cmd: "import-queue", send: true, en: "Import the queue", pt: "Importar a fila" },
|
|
312
314
|
{ cmd: "history", send: true, en: "Recent changes", pt: "Alterações recentes" },
|
|
313
315
|
{ cmd: "undo", send: true, en: "Undo the last change", pt: "Desfazer a última alteração" },
|
|
316
|
+
{ cmd: "feedback", en: "Report a problem", pt: "Comunicar um problema" },
|
|
314
317
|
],
|
|
315
318
|
};
|
|
316
319
|
|
|
@@ -332,6 +335,7 @@ export const SUGGESTIONS = {
|
|
|
332
335
|
export const TOOLS = {
|
|
333
336
|
en: {
|
|
334
337
|
find_items: "search the library",
|
|
338
|
+
get_figure: "cut a figure out of the PDF",
|
|
335
339
|
standalone_items: "list files without a parent item",
|
|
336
340
|
set_parent_items: "give files a parent item",
|
|
337
341
|
find_reference: "look up a reference",
|
|
@@ -392,6 +396,7 @@ export const TOOLS = {
|
|
|
392
396
|
},
|
|
393
397
|
pt: {
|
|
394
398
|
find_items: "pesquisar na biblioteca",
|
|
399
|
+
get_figure: "recortar uma figura do PDF",
|
|
395
400
|
standalone_items: "listar ficheiros sem item principal",
|
|
396
401
|
set_parent_items: "dar um item principal aos ficheiros",
|
|
397
402
|
find_reference: "procurar uma referência",
|