fh-library 0.2.11 → 0.2.13

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/CHANGELOG.md CHANGED
@@ -1,7 +1,15 @@
1
1
  # FH Library — Change Log
2
2
 
3
+ ### v0.2.12 — 2026-11-08
4
+
5
+ * **`library_pubmed_search_articles(query, max_results?, sort?, years?)`:** a PubMed search in PubMed's own syntax — esearch for the total and the IDs, one esummary for the rows (PMID, title, first author, year, journal, publication types, PMCID, library level), cached under `search/`. With the batch fetch and `library_read` this covers what the AI4L models used of the PubMed MCP server (#27).
6
+ * **`FH_LIBRARY_SEARCH_CACHE_DAYS` (7):** days a PubMed or registry search is cached; was a constant.
7
+ * **An ID no service knows is remembered** for `FH_LIBRARY_RETRY_AFTER_DAYS` (1): a PMID PubMed has no record for, a DOI neither Crossref nor doi.org resolves, a set ID without a label, an NCT ID without a study. A marker (`<entry>-missing.md`, date and reason, no meta) in the entry's directory; `library_add`, `library_status`, `library_read`, `library_verify` and the batch tool answer from it ("pmid not found (checked 2026-11-08; asked again after 1 day)"), the service is asked again after the cooldown, a real entry replaces the marker. Network errors are never remembered. The same cooldown a failed page or a transient tier failure already had.
8
+ * **`library_pubmed_fetch_articles(pmids)`:** title, first author, year, publication types, retracted and level for up to 200 PMIDs in one call, rows in `columns` order (~120 characters a paper against the PubMed MCP server's ~2,500). Stored PMIDs from disk; the rest in one efetch, stored at abstract level as `library_add`'s first stage, so the cascade runs later as on an untried entry. The first of the PubMed tools (#27); `library_info`'s `cache` and `.servers/` count `pubmed` hits and misses per PMID.
9
+
3
10
  ### v0.2.9 — 2026-11-08
4
11
 
12
+ * TUI v82: the `url` row shows library_fetch hits and misses from the servers' counters; `/status` carries `fetches`. Counts right-aligned in a 5-character column.
5
13
  * TUI v80: a `search` row — cached registry searches, hits and misses from the servers' counters; the decision-model row is labelled `decider`; `/status` carries `searches`.
6
14
  * ClinicalTrials tool descriptions and server instructions one statement each.
7
15
 
package/README.md CHANGED
@@ -1,4 +1,4 @@
1
- ![Version 0.1.3](https://img.shields.io/badge/Version-0.2.11-green.svg)
1
+ ![Version 0.1.3](https://img.shields.io/badge/Version-0.2.13-green.svg)
2
2
  ![License MIT](https://img.shields.io/badge/License-MIT-yellow.svg)
3
3
  [![Forever Healthy](https://img.shields.io/badge/(c)_2026-Forever_Healthy-573D7D.svg)](https://forever-healthy.org)
4
4
  [![npm](https://img.shields.io/badge/npm-fh--library-cb3837.svg)](https://www.npmjs.com/package/fh-library)
@@ -23,6 +23,8 @@ A [Model Context Protocol](https://modelcontextprotocol.io) server that builds a
23
23
  * `library_info(ids?)` → the whole library, everything the TUI shows: the library directory and host, the decision model with its endpoint and whether it answers right now (`verify`), this server's cache hits and misses since it started (`cache`), the entries and their size on disk (`size`), per entry type (`types`: pmid, doi, dailymed, nct) the entries with their levels — `full_text`, `abstract`, `title` — and retractions; entry counts by type and level, counts of retracted and not-citable entries and of entries with errata, expressions of concern and updates, the configured tiers, library directory and host; `ids: true` also lists every entry ID
24
24
  * `library_verify(claim, id)` → is one finding of a review stated by one paper it cites? `claim` is the finding alone, verbatim, without its citations; `id` one paper (`pmid:`, or any `doi:`), drug label or trial record the review cites for it — a finding cited to three papers is three calls with the same claim; a claim that names a source or carries citations is refused. The server reads the paper (full text, else abstract) and asks a decision model; nothing of the paper enters the model's context. `verdict` **pass** (`score` the probability), **fail** (the full text does not state it; or only the title is in the library — `reason` says so), **unverifiable** (only the abstract is in the library and does not state it — `reason`) or **error**; `basis` full_text, abstract or title. See "Verifying claims" below.
25
25
  * `library_read(id, section?, query?, max_chars?, max_hits?)` → one small piece of a stored paper's Markdown, never the whole paper: without `section`/`query` the **outline** (headings with word counts, and whether the full text or only the abstract is stored; a heading the paper uses several times is listed as `Participants [3/12]`); `section` → that section's **own text** (heading matched as case-insensitive substring, capped at `max_chars`, default 6000, with `returned_words` against `words` when cut) and, for a parent heading, its `subsections` with word counts to pick from — a duplicated heading named bare lists the candidates instead; `query` — two or three distinctive terms the paper itself would use (a measure and a unit, an outcome and a group, a compound name), not the claim — → **passages** holding every term, each a window of at most 1000 chars around the matches (`clipped: true` when cut; a table row with its header row), with its section, best first (at most `max_hits`, default 5); matching forgives case, accents, hyphens, Markdown markup, plural and -ed/-ing forms; when no passage holds every term the best **partial** matches come back (`partial: true`, the `matched` terms per passage, the rare term first), and `missing_terms` names the terms the text never uses in any form — those are the ones to rephrase. `section` and `query` together search inside that section. `section: "Abstract"` on a full text without one answers from the stored abstract. Every full text converted from JATS XML (PMC, Europe PMC, Elsevier) has a **Declarations** section — funding, conflicts of interest, author notes, gathered from wherever the XML keeps them — and on a text without one `section: "Declarations"` assembles the sections headed funding, conflicts, acknowledgements, ethics, contributions, so it answers who paid and who declares what on every source alike; a paper with neither states nothing. Outline first, then pick. A paper not in the library is added first, as `library_add` would (its lines in the log), then library_read; the answer carries `added: true`. Reads may run in parallel.
26
+ * `library_pubmed_search_articles(query, max_results?, sort?, years?)` → a PubMed search in PubMed's own syntax (see "PubMed in batches"): the total and rows in `columns` order, with the library level of a paper already stored, cached for `FH_LIBRARY_SEARCH_CACHE_DAYS`
27
+ * `library_pubmed_fetch_articles(pmids)` → title, first author, year, publication types, retracted and level for up to 200 PMIDs in one call (see "PubMed in batches"); PMIDs the library lacks are fetched in one PubMed request and stored at abstract level
26
28
  * `library_clinicaltrials_search_studies(query | intervention | condition, status?, sort?, max_results?)` → a ClinicalTrials.gov search (see "Trial search"): the total count and compact rows, cached for a week
27
29
  * `library_clinicaltrials_get_study_record(nct)` → one trial as a review needs it (title, status, phase, design, enrollment, dates, sponsor, conditions, interventions, primary outcomes, eligibility, sites, result publications, has_results, brief summary), from the stored record
28
30
  * `library_clinicaltrials_get_study_results(nct, max_chars?)` → a trial's posted results as Markdown, capped, with the outcome measure titles for `library_read`
@@ -58,7 +60,7 @@ Downloads are plain HTTP with a desktop Chrome User-Agent (API calls identify th
58
60
 
59
61
  * **Which tiers run, in which order:** `FH_LIBRARY_TIERS=pmc,tdm,unpaywall,openalex` — exactly these, in this order; a tier not listed is off. Unset, every tier runs in the order of the table above, the proxy last.
60
62
  * **Keys:** `FH_LIBRARY_ELSEVIER_API_KEY` and `FH_LIBRARY_WILEY_TDM_TOKEN` for `tdm`, `FH_LIBRARY_CORE_API_KEY` for `core`, `FH_LIBRARY_NCBI_API_KEY` (10 instead of 3 requests/s), `FH_LIBRARY_BRIGHTDATA_API_TOKEN` and `FH_LIBRARY_BRIGHTDATA_ZONE` for the unlocker. `FH_LIBRARY_CONTACT_EMAIL` is the one address sent to every service (NCBI `tool`/`email`, Unpaywall and OpenAlex polite pools, Crossref, the User-Agent).
61
- * **Retries:** `FH_LIBRARY_RETRY_AFTER_DAYS` (default 1) is the cooldown before a transient tier failure is retried; `FH_LIBRARY_REFRESH_AFTER_DAYS` (30) the age of an entry's PubMed record after which `library_add` re-reads it for new correction notices and retractions. A tdm link behind a token that is not configured is recorded as `needs elsevier token` and not re-checked; after adding the token, remove those attempts from the entries to give them a go. `FH_LIBRARY_RETRY_BLOCKED` (false) decides whether a browser challenge that did not clear, or a Bright Data refusal, is final: with the same method tomorrow ends the same way, so set it true only after a change of method (a new browser rung, a new service).
63
+ * **Retries:** `FH_LIBRARY_RETRY_AFTER_DAYS` (default 1) is the cooldown before a transient tier failure is retried, before a page `library_fetch` could not get is fetched again, and before an ID no service knew (a PMID, DOI, set ID or NCT ID that is not found) is asked for again — meanwhile every tool answers "not found" from a marker in the entry's directory, with the date it was checked; `FH_LIBRARY_SEARCH_CACHE_DAYS` (7) how long a PubMed or registry search is answered from disk; `FH_LIBRARY_REFRESH_AFTER_DAYS` (30) the age of an entry's PubMed record after which `library_add` re-reads it for new correction notices and retractions. A tdm link behind a token that is not configured is recorded as `needs elsevier token` and not re-checked; after adding the token, remove those attempts from the entries to give them a go. `FH_LIBRARY_RETRY_BLOCKED` (false) decides whether a browser challenge that did not clear, or a Bright Data refusal, is final: with the same method tomorrow ends the same way, so set it true only after a change of method (a new browser rung, a new service).
62
64
  * **Browsers and verify:** `FH_LIBRARY_PLAYWRIGHT_n` and `FH_LIBRARY_CHROME_n` (hostnames; above) and `FH_LIBRARY_PLAYWRIGHT_HEADLESS`; `FH_LIBRARY_VERIFY_*` ("Verifying claims"). A Playwright server must run the **same `playwright-core` version** as the package — `library_info` shows the required one as `download.playwright.version`, `browser/Dockerfile` builds a server at it, and a server of another version is refused by Playwright itself (the log then carries the alert `!! browser <host>: Playwright version mismatch — server … must run playwright-core <version>`). Moving the version is one release: the pin in `package.json`, the tag in `browser/Dockerfile`, every server rebuilt. The library never launches a browser itself: every rung is a browser somebody runs.
63
65
 
64
66
  `library_info` reports the tiers in run order and, per variable, whether its value came from the environment, `.env` or the default — never the value.
@@ -147,7 +149,9 @@ PubMed's book records — National Academies reports, NTP monographs, StatPearls
147
149
 
148
150
  **Clinical trial records.** `library_add("nct:05173896")` stores the current record of one trial from [ClinicalTrials.gov](https://clinicaltrials.gov/): the ID is the eight digits of the NCT number in the review's link (`…/study/NCT05173896`); `nct:NCT05173896` and a bare `NCT05173896` are accepted and read the same. The API's JSON is converted to Markdown, one section per module: identification with secondary IDs such as EudraCT, status with every date and its actual/estimated type, sponsor and collaborators, description, conditions, design (phase, allocation, masking, enrollment with its type), arms and interventions with doses, primary and secondary outcomes with time frames, eligibility, locations as a country list, references; and, when results are posted, the participant flow, baseline characteristics, every outcome measure with its group values and analyses, and the adverse events, all as tables. Values are written in plain words, "Phase 2", "Active, not recruiting", "100 (estimated)", so a figure the review states can be found as stated. The status carries `nctid`, `publications` (the results publications the record lists, as `pmid:…`, so whether a trial has been published is one status away), the brief title, the lead sponsor as `first_author`, the first-posted year as `year`, the last-update-posted date as `version`, and `publication_types` `["Trial Registration"]`. `library_read` and `library_verify` work on a record as on a paper. An entry older than `FH_LIBRARY_REFRESH_AFTER_DAYS` is re-fetched and rewritten when the record was updated since; a status the review states that has moved on then fails, which is right. Entries live under `nct/<digits 1–2>/<digits 1–4>/<digits>/` with the JSON and the Markdown as `nct-<digits>-fulltext.json` and `-fulltext.md`; the log line reads `library_add nct:05173896 ctgov fetched level=full_text updated 2024-12-09 active, not recruiting ETLAS2`.
149
151
 
150
- **Trial search.** `library_clinicaltrials_search_studies(query?, intervention?, condition?, status?, sort?, max_results?)` asks the ClinicalTrials.gov API (v2, no key) what a review needs from the registry: the trials of an intervention, the ongoing ones (`status: "ongoing"` is recruiting, not yet recruiting, active not recruiting, enrolling by invitation), a free-text or condition search, sorted by relevance, newest, largest or completion date, up to 100 rows. The rows are arrays in the order of `columns` — the field names once, not per row — shaped for the review's Emerging Research item (what AI4L's prompt asks about an ongoing trial; the link is `https://clinicaltrials.gov/study/<NCT ID>`): NCT ID, title, status with why a trial stopped, phase, enrollment as "100 (estimated)", start and primary completion dates (whether "ongoing" still holds), the primary outcome with its time frame, conditions, interventions, `has_results`; the full detail of one row is `library_clinicaltrials_get_study_record`; `total` is the registry's count, for a "no trials found" statement. The same search is answered from `search/<2>/<4>/<key>.json` for 7 days — a registry listing a week old is current enough for a review, and an audit that runs again reads from disk. Two thin tools answer from the stored record of any row (added first when missing): `library_clinicaltrials_get_study_record(nct)` is the review's detail of one trial — official title, design, enrollment, dates, eligibility, sites, result publications as `pmid:` IDs, the brief summary — and `library_clinicaltrials_get_study_results(nct, max_chars?)` is the posted results part of the stored Markdown (participant flow, baseline, outcome measures, adverse events), with the outcome titles listed so `library_read("nct:…", section)` can return one measure whole. Together with the search they cover what the AI4L models asked the ClinicalTrials.gov MCP server for.
152
+ **PubMed in batches.** `library_pubmed_fetch_articles(pmids)` answers what an auditor asks of a list of PubMed IDs — does each resolve, what is its title, year and type, is it retracted, does the library hold its full text — as rows in `columns` order, about 120 characters per paper. A PMID already in the library is read from disk. The others go to PubMed in one efetch request (up to 200), and each record is stored exactly as `library_add` stores its first stage: the XML, the abstract Markdown, the meta with its correction and retraction notices, no retrieval attempts yet — so a later `library_add` or `library_read` runs the full-text cascade as on an untried entry, and the library fills at abstract level in batches instead of one paper at a time. An entry is checked again right before writing, so a concurrent `library_add` that already stored a full text is never overwritten. Level `not found` marks a PMID PubMed does not know, `invalid` a value that is not a PMID. `library_pubmed_search_articles(query, max_results?, sort?, years?)` is the search: the query goes to PubMed as written, field tags and all; esearch gives the total and the IDs, one esummary the rows — PMID, title, first author, year, journal, publication types (Journal Article left out), PMCID when a free full text exists, and the library's level when the paper is already stored, so the model knows what `library_read` can answer at once. Cached under `search/` like the trial search.
153
+
154
+ **Trial search.** `library_clinicaltrials_search_studies(query?, intervention?, condition?, status?, sort?, max_results?)` asks the ClinicalTrials.gov API (v2, no key) what a review needs from the registry: the trials of an intervention, the ongoing ones (`status: "ongoing"` is recruiting, not yet recruiting, active not recruiting, enrolling by invitation), a free-text or condition search, sorted by relevance, newest, largest or completion date, up to 100 rows. The rows are arrays in the order of `columns` — the field names once, not per row — shaped for the review's Emerging Research item (what AI4L's prompt asks about an ongoing trial; the link is `https://clinicaltrials.gov/study/<NCT ID>`): NCT ID, title, status with why a trial stopped, phase, enrollment as "100 (estimated)", start and primary completion dates (whether "ongoing" still holds), the primary outcome with its time frame, conditions, interventions, `has_results`; the full detail of one row is `library_clinicaltrials_get_study_record`; `total` is the registry's count, for a "no trials found" statement. The same search is answered from `search/<2>/<4>/<key>.json` for `FH_LIBRARY_SEARCH_CACHE_DAYS` (7) — a registry listing a week old is current enough for a review, and an audit that runs again reads from disk. Two thin tools answer from the stored record of any row (added first when missing): `library_clinicaltrials_get_study_record(nct)` is the review's detail of one trial — official title, design, enrollment, dates, eligibility, sites, result publications as `pmid:` IDs, the brief summary — and `library_clinicaltrials_get_study_results(nct, max_chars?)` is the posted results part of the stored Markdown (participant flow, baseline, outcome measures, adverse events), with the outcome titles listed so `library_read("nct:…", section)` can return one measure whole. Together with the search they cover what the AI4L models asked the ClinicalTrials.gov MCP server for.
151
155
 
152
156
  **Web pages.** `library_fetch(url, max_length?, start_index?, raw?)` is the standard MCP [fetch](https://github.com/modelcontextprotocol/servers/tree/main/src/fetch) tool, served from the library, for every link a review carries that is not a paper, label or trial record: a guideline, a product page, a news item. Its description tells the model to prefer it over a plain fetch or a browser tool, and says why: a page anyone fetched in the last 30 days comes from disk in milliseconds, and a page a plain fetch cannot get climbs the ladder in the same call. The answer is what `fetch` returns, so a model keeps its habits: `Contents of <url>:` and the page as Markdown with the page's title as the first heading, cut at `max_length` (default 5000) with the note `Content truncated. Call the fetch tool with a start_index of N to get more content`; `raw` gives the HTML; a `text/markdown` body comes back as is, any other non-HTML body with the `Content type … cannot be simplified to markdown` prefix; a failure is `Failed to fetch <url> - status code 404` or `Failed to fetch <url>: <reason>`. Two things happen underneath. The page is fetched up the same ladder as a paper — the plain fetch, then the browser rungs on a 403, 429 or bot wall, then Bright Data — so a page a plain fetch cannot reach is still read. And it is stored: `url/<2>/<4>/<16 hex of the URL's SHA-1>/url-<hash>-{meta.md, page.md, page.html|pdf|txt}`, the URL normalised first (host lowercased, fragment and tracking parameters dropped). The same URL within `FH_LIBRARY_REFRESH_AFTER_DAYS` (30) is served from disk, a failed fetch is not repeated within `FH_LIBRARY_RETRY_AFTER_DAYS` (1) — an audit that runs twice over one review reads its links once. A PDF comes back as Markdown (`fetch` would return its bytes as text). `title_only` returns `Contents of <url>:` and the title line alone — the page is still fetched and stored in full — so a link check costs a few tokens instead of a page. The log says which — `fetch example.com/… cached html … https://example.com/drugs/x` or `fetched 02.3s html retrieved_by=browser hetzner … <url>` (the ID column holds the URL without scheme, cut to a PubMed ID's width; the full URL ends the line); `library_info` counts the pages under `url`, a failed fetch as level `title`.
153
157
 
package/dist/index.js CHANGED
@@ -1725,6 +1725,7 @@ var VARIABLES = [
1725
1725
  "FH_LIBRARY_RETRY_AFTER_DAYS",
1726
1726
  "FH_LIBRARY_RETRY_BLOCKED",
1727
1727
  "FH_LIBRARY_REFRESH_AFTER_DAYS",
1728
+ "FH_LIBRARY_SEARCH_CACHE_DAYS",
1728
1729
  "FH_LIBRARY_NCBI_API_KEY",
1729
1730
  "FH_LIBRARY_BRIGHTDATA_API_TOKEN",
1730
1731
  "FH_LIBRARY_BRIGHTDATA_ZONE"
@@ -1734,7 +1735,8 @@ function settings(env = process.env) {
1734
1735
  return {
1735
1736
  retry_after_days: num("FH_LIBRARY_RETRY_AFTER_DAYS", 1, env),
1736
1737
  retry_blocked: bool("FH_LIBRARY_RETRY_BLOCKED", false, env),
1737
- refresh_after_days: num("FH_LIBRARY_REFRESH_AFTER_DAYS", 30, env)
1738
+ refresh_after_days: num("FH_LIBRARY_REFRESH_AFTER_DAYS", 30, env),
1739
+ search_cache_days: num("FH_LIBRARY_SEARCH_CACHE_DAYS", 7, env)
1738
1740
  };
1739
1741
  }
1740
1742
  function proxyHeaders(spec) {
@@ -1809,11 +1811,11 @@ function log(tool, id, message) {
1809
1811
  }
1810
1812
 
1811
1813
  // src/stats.ts
1812
- import { mkdirSync as mkdirSync4, readdirSync as readdirSync3, readFileSync as readFileSync5, statSync as statSync2, unlinkSync as unlinkSync2 } from "fs";
1814
+ import { mkdirSync as mkdirSync4, readdirSync as readdirSync3, readFileSync as readFileSync5, statSync as statSync2, unlinkSync as unlinkSync3 } from "fs";
1813
1815
  import { join as join7 } from "path";
1814
1816
 
1815
1817
  // src/entry.ts
1816
- import { copyFileSync, existsSync as existsSync5, mkdirSync as mkdirSync3, readdirSync as readdirSync2, readFileSync as readFileSync4, renameSync as renameSync2, rmSync, writeFileSync } from "fs";
1818
+ import { copyFileSync, existsSync as existsSync5, mkdirSync as mkdirSync3, readdirSync as readdirSync2, readFileSync as readFileSync4, renameSync as renameSync2, rmSync, unlinkSync as unlinkSync2, writeFileSync } from "fs";
1817
1819
  import { join as join6 } from "path";
1818
1820
  import { basename, dirname } from "path";
1819
1821
 
@@ -2017,11 +2019,11 @@ function inline(nodes) {
2017
2019
  return out;
2018
2020
  }
2019
2021
  function table(n) {
2020
- const row2 = (x) => kids2(x).filter((c) => ["td", "th"].includes(tagOf2(c))).map((c) => squash2(inline(kids2(c))).replace(/\|/g, "\\|"));
2022
+ const row3 = (x) => kids2(x).filter((c) => ["td", "th"].includes(tagOf2(c))).map((c) => squash2(inline(kids2(c))).replace(/\|/g, "\\|"));
2021
2023
  const collect = (nodes, into) => {
2022
2024
  for (const x of nodes) {
2023
2025
  if ("#text" in x) continue;
2024
- if (tagOf2(x) === "tr") into.push(row2(x));
2026
+ if (tagOf2(x) === "tr") into.push(row3(x));
2025
2027
  else if (!["thead", "tfoot"].includes(tagOf2(x))) collect(kids2(x), into);
2026
2028
  }
2027
2029
  };
@@ -2030,7 +2032,7 @@ function table(n) {
2030
2032
  const t = tagOf2(x);
2031
2033
  if (t === "thead") collect(kids2(x), head);
2032
2034
  else if (t === "tfoot") collect(kids2(x), foot);
2033
- else if (t === "tr") body.push(row2(x));
2035
+ else if (t === "tr") body.push(row3(x));
2034
2036
  else if (!("#text" in x) && t !== "caption") collect(kids2(x), body);
2035
2037
  }
2036
2038
  const rows = [...head, ...body];
@@ -2636,6 +2638,10 @@ function writeAtomic(file2, data) {
2636
2638
  }
2637
2639
  function writeMeta(p, m) {
2638
2640
  writeAtomic(p.meta, renderMeta(m));
2641
+ try {
2642
+ unlinkSync2(join6(p.entryDir, `${p.entry}-missing.md`));
2643
+ } catch {
2644
+ }
2639
2645
  }
2640
2646
  function copyEntry(from, to) {
2641
2647
  const tmp = `${to.entryDir}.${process.pid}.tmp`, old = `${to.entryDir}.${process.pid}.old`;
@@ -2917,14 +2923,14 @@ var STALE_MS = 20 * 6e4;
2917
2923
  var stats = null;
2918
2924
  var file = null;
2919
2925
  function startStats(pkg2, libraryDir = FH_LIBRARY_DIR) {
2920
- stats = { host: HOST, pid: process.pid, package: pkg2.name, version: pkg2.version, started: (/* @__PURE__ */ new Date()).toISOString(), verify: { hits: 0, misses: 0 }, fetch: { hits: 0, misses: 0 }, search: { hits: 0, misses: 0 } };
2926
+ stats = { host: HOST, pid: process.pid, package: pkg2.name, version: pkg2.version, started: (/* @__PURE__ */ new Date()).toISOString(), verify: { hits: 0, misses: 0 }, fetch: { hits: 0, misses: 0 }, search: { hits: 0, misses: 0 }, pubmed: { hits: 0, misses: 0 } };
2921
2927
  file = join7(libraryDir, ".servers", `${HOST}-${process.pid}.json`);
2922
2928
  sweep(join7(libraryDir, ".servers"));
2923
2929
  publish();
2924
2930
  setInterval(publish, HEARTBEAT_MS).unref();
2925
2931
  const stop = () => {
2926
2932
  try {
2927
- if (file) unlinkSync2(file);
2933
+ if (file) unlinkSync3(file);
2928
2934
  } catch {
2929
2935
  }
2930
2936
  };
@@ -2945,7 +2951,7 @@ function sweep(dir, staleMs = STALE_MS) {
2945
2951
  const p = join7(dir, name);
2946
2952
  try {
2947
2953
  if (Date.now() - statSync2(p).mtimeMs > staleMs) {
2948
- unlinkSync2(p);
2954
+ unlinkSync3(p);
2949
2955
  n++;
2950
2956
  }
2951
2957
  } catch {
@@ -2953,9 +2959,9 @@ function sweep(dir, staleMs = STALE_MS) {
2953
2959
  }
2954
2960
  return n;
2955
2961
  }
2956
- function bump(kind, hit) {
2962
+ function bump(kind, hit, n = 1) {
2957
2963
  if (!stats) return;
2958
- stats[kind][hit ? "hits" : "misses"]++;
2964
+ stats[kind][hit ? "hits" : "misses"] += n;
2959
2965
  publish();
2960
2966
  }
2961
2967
  var currentStats = () => stats;
@@ -2969,7 +2975,7 @@ function publish() {
2969
2975
  }
2970
2976
 
2971
2977
  // src/tools.ts
2972
- import { existsSync as existsSync11, readFileSync as readFileSync11 } from "fs";
2978
+ import { existsSync as existsSync13, readFileSync as readFileSync13 } from "fs";
2973
2979
  import { z } from "zod";
2974
2980
 
2975
2981
  // src/cascade.ts
@@ -3999,7 +4005,7 @@ async function verifyCached(claim, id, cfg, decide, libraryDir = FH_LIBRARY_DIR,
3999
4005
  var DROP = /* @__PURE__ */ new Set(["entry_dir", "library_dir", "files"]);
4000
4006
  var compact2 = (o) => JSON.stringify(o);
4001
4007
  function lean(o) {
4002
- if (Array.isArray(o)) return o.map((row2) => row2 && typeof row2 === "object" && !Array.isArray(row2) ? leanObject(row2) : row2);
4008
+ if (Array.isArray(o)) return o.map((row3) => row3 && typeof row3 === "object" && !Array.isArray(row3) ? leanObject(row3) : row3);
4003
4009
  if (o && typeof o === "object") return leanObject(o);
4004
4010
  return o;
4005
4011
  }
@@ -4017,7 +4023,7 @@ var leanJson = (o) => compact2(lean(o));
4017
4023
  import { createHash as createHash4 } from "crypto";
4018
4024
  import { existsSync as existsSync10, mkdirSync as mkdirSync7, readFileSync as readFileSync10 } from "fs";
4019
4025
  import { dirname as dirname4, join as join11 } from "path";
4020
- var SEARCH_AFTER_DAYS = 7;
4026
+ var searchCacheDays = () => settings().search_cache_days;
4021
4027
  var API2 = "https://clinicaltrials.gov/api/v2/studies";
4022
4028
  var FIELDS = ["NCTId", "BriefTitle", "OverallStatus", "WhyStopped", "Phase", "StudyType", "EnrollmentCount", "EnrollmentType", "StartDate", "PrimaryCompletionDate", "CompletionDate", "LastUpdatePostDate", "PrimaryOutcomeMeasure", "PrimaryOutcomeTimeFrame", "Condition", "InterventionName", "LeadSponsorName", "HasResults"];
4023
4029
  var ONGOING = ["RECRUITING", "NOT_YET_RECRUITING", "ACTIVE_NOT_RECRUITING", "ENROLLING_BY_INVITATION"];
@@ -4109,7 +4115,7 @@ async function searchTrials(raw, libraryDir = FH_LIBRARY_DIR, fetchImpl = liveFe
4109
4115
  if (existsSync10(file2)) {
4110
4116
  try {
4111
4117
  const r = JSON.parse(readFileSync10(file2, "utf8"));
4112
- if (daysSince(r.searched) < SEARCH_AFTER_DAYS) return { ok: true, result: r, cached: true };
4118
+ if (daysSince(r.searched) < searchCacheDays()) return { ok: true, result: r, cached: true };
4113
4119
  } catch {
4114
4120
  }
4115
4121
  }
@@ -4138,6 +4144,123 @@ function trialsLogId(q) {
4138
4144
  return s.length > 10 ? s.slice(0, 9) + "\u2026" : s || "-";
4139
4145
  }
4140
4146
 
4147
+ // src/pubmedsearch.ts
4148
+ import { createHash as createHash5 } from "crypto";
4149
+ import { existsSync as existsSync11, mkdirSync as mkdirSync8, readFileSync as readFileSync11 } from "fs";
4150
+ import { dirname as dirname5 } from "path";
4151
+ var MAX_RESULTS2 = 100;
4152
+ var DEFAULT_RESULTS2 = 20;
4153
+ var COLUMNS2 = ["pmid", "title", "first_author", "year", "journal", "publication_types", "pmcid", "level"];
4154
+ var EUTILS = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils";
4155
+ var ident = (u) => {
4156
+ if (process.env.FH_LIBRARY_NCBI_API_KEY) u.searchParams.set("api_key", process.env.FH_LIBRARY_NCBI_API_KEY);
4157
+ u.searchParams.set("tool", "fh-library");
4158
+ u.searchParams.set("email", contactEmail());
4159
+ return u.toString();
4160
+ };
4161
+ function yearRange(years) {
4162
+ const t = (years ?? "").replace(/\s+/g, "");
4163
+ if (!t) return null;
4164
+ const m = /^(\d{4})?(-)?(\d{4})?$/.exec(t);
4165
+ if (!m || !m[1] && !m[3]) return { error: `years must be "2020-2026", "2022-", "-2019" or "2022", not "${years}"` };
4166
+ const min = m[1] ?? "1000", max = m[3] ?? (m[2] ? String((/* @__PURE__ */ new Date()).getFullYear()) : m[1]);
4167
+ return { min, max };
4168
+ }
4169
+ function normalizeQuery2(q) {
4170
+ const query = (q.query ?? "").replace(/\s+/g, " ").trim();
4171
+ if (!query) return { error: "empty query" };
4172
+ const sort = q.sort ?? "relevance";
4173
+ if (!["relevance", "newest"].includes(sort)) return { error: `sort must be relevance or newest, not "${String(q.sort)}"` };
4174
+ const yr = yearRange(q.years);
4175
+ if (yr && "error" in yr) return yr;
4176
+ const out = { query, sort, max_results: Math.min(MAX_RESULTS2, Math.max(1, Math.floor(q.max_results ?? DEFAULT_RESULTS2))) };
4177
+ if (yr) out.years = `${yr.min === "1000" ? "" : yr.min}-${yr.max}`;
4178
+ return out;
4179
+ }
4180
+ var searchKey2 = (q) => createHash5("sha1").update(JSON.stringify(["pubmed", q.query, q.sort, q.years ?? "", q.max_results])).digest("hex").slice(0, 16);
4181
+ function esearchUrl(q) {
4182
+ const u = new URL(`${EUTILS}/esearch.fcgi`);
4183
+ u.searchParams.set("db", "pubmed");
4184
+ u.searchParams.set("term", q.query);
4185
+ u.searchParams.set("retmax", String(q.max_results));
4186
+ u.searchParams.set("retmode", "json");
4187
+ u.searchParams.set("sort", q.sort === "newest" ? "pub_date" : "relevance");
4188
+ const yr = yearRange(q.years);
4189
+ if (yr && !("error" in yr)) {
4190
+ u.searchParams.set("datetype", "pdat");
4191
+ u.searchParams.set("mindate", yr.min);
4192
+ u.searchParams.set("maxdate", yr.max);
4193
+ }
4194
+ return ident(u);
4195
+ }
4196
+ function esummaryUrl(pmids) {
4197
+ const u = new URL(`${EUTILS}/esummary.fcgi`);
4198
+ u.searchParams.set("db", "pubmed");
4199
+ u.searchParams.set("id", pmids.join(","));
4200
+ u.searchParams.set("retmode", "json");
4201
+ return ident(u);
4202
+ }
4203
+ function parseEsearch(json2) {
4204
+ const d = JSON.parse(json2);
4205
+ const r = d?.esearchresult ?? {};
4206
+ return { total: Number(r.count ?? 0) || 0, ids: Array.isArray(r.idlist) ? r.idlist.map(String) : [] };
4207
+ }
4208
+ function parseEsummary(json2, ids, libraryDir) {
4209
+ const d = JSON.parse(json2);
4210
+ const res = d?.result ?? {};
4211
+ const rows = [];
4212
+ for (const id of ids) {
4213
+ const s = res[id];
4214
+ if (!s || s.error) continue;
4215
+ const year = /\b(\d{4})\b/.exec(String(s.pubdate ?? ""))?.[1] ?? null;
4216
+ const first = String(s.sortfirstauthor ?? (Array.isArray(s.authors) && s.authors[0]?.name) ?? "").replace(/\s+[A-Z]{1,3}$/, "").trim() || null;
4217
+ const pmc2 = (Array.isArray(s.articleids) ? s.articleids : []).find((a) => a.idtype === "pmc")?.value ?? null;
4218
+ const types = Array.isArray(s.pubtype) ? s.pubtype.filter((t) => t !== "Journal Article").join("; ") : "";
4219
+ const paths = entryPaths(libraryDir, "pmid", id);
4220
+ const level = entryExists(paths) ? String(readStatus(paths, emptyStatus(`pmid:${id}`, "pmid", id, libraryDir)).level) : null;
4221
+ rows.push([id, String(s.title ?? "").replace(/\.$/, "") || null, first, year, s.source ? String(s.source) : null, types || null, pmc2 ? String(pmc2) : null, level]);
4222
+ }
4223
+ return rows;
4224
+ }
4225
+ var live = (url) => fetchText(url, { "User-Agent": "fh-library", Accept: "application/json" });
4226
+ async function searchArticles(raw, libraryDir = FH_LIBRARY_DIR, fetchImpl = live) {
4227
+ const q = normalizeQuery2(raw);
4228
+ if ("error" in q) return { ok: false, error: q.error };
4229
+ const file2 = searchFile(libraryDir, searchKey2(q));
4230
+ if (existsSync11(file2)) {
4231
+ try {
4232
+ const r = JSON.parse(readFileSync11(file2, "utf8"));
4233
+ if (daysSince(r.searched) < searchCacheDays()) return { ok: true, result: r, cached: true };
4234
+ } catch {
4235
+ }
4236
+ }
4237
+ let total, ids;
4238
+ try {
4239
+ ({ total, ids } = parseEsearch(await fetchImpl(esearchUrl(q))));
4240
+ } catch (e) {
4241
+ return { ok: false, error: `pubmed: ${e instanceof Error ? e.message : String(e)}` };
4242
+ }
4243
+ let rows = [];
4244
+ if (ids.length) {
4245
+ try {
4246
+ rows = parseEsummary(await fetchImpl(esummaryUrl(ids)), ids, libraryDir);
4247
+ } catch (e) {
4248
+ return { ok: false, error: `pubmed: ${e instanceof Error ? e.message : String(e)}` };
4249
+ }
4250
+ }
4251
+ const result = { source: "pubmed", query: q, total, columns: COLUMNS2, rows, searched: today() };
4252
+ try {
4253
+ mkdirSync8(dirname5(file2), { recursive: true });
4254
+ writeAtomic(file2, JSON.stringify(result, null, 2) + "\n");
4255
+ } catch {
4256
+ }
4257
+ return { ok: true, result, cached: false };
4258
+ }
4259
+ function logId2(q) {
4260
+ const s = q.query.replace(/\s+/g, " ");
4261
+ return s.length > 10 ? s.slice(0, 9) + "\u2026" : s;
4262
+ }
4263
+
4141
4264
  // src/trialrecord.ts
4142
4265
  var list3 = (a) => Array.isArray(a) ? a : [];
4143
4266
  var para2 = (s, max = Infinity) => {
@@ -4209,10 +4332,120 @@ function resultsText(nct, md, maxChars) {
4209
4332
  return { nct, has_results: true, text: truncated ? part.slice(0, maxChars).trimEnd() + "\n\n[truncated]" : part, truncated, outcomes, note: truncated ? "cut at max_chars; library_read(id, section: <an outcome title>) returns one measure whole" : null };
4210
4333
  }
4211
4334
 
4335
+ // src/missing.ts
4336
+ import { existsSync as existsSync12, mkdirSync as mkdirSync9, readFileSync as readFileSync12 } from "fs";
4337
+ import { join as join12 } from "path";
4338
+ var missingFile = (p) => join12(p.entryDir, `${p.entry}-missing.md`);
4339
+ function readMissing(p) {
4340
+ const f = missingFile(p);
4341
+ if (!existsSync12(f)) return null;
4342
+ const md = readFileSync12(f, "utf8");
4343
+ const get = (k) => /^$/.test(k) ? "" : (new RegExp(`^${k}:[ \\t]*(.*)$`, "m").exec(md)?.[1] ?? "").trim();
4344
+ const checked = get("checked"), reason = get("reason");
4345
+ return checked ? { checked, reason: reason || "not found" } : null;
4346
+ }
4347
+ function freshMissing(p, retryDays) {
4348
+ const m = readMissing(p);
4349
+ return m && daysSince(m.checked) < retryDays ? m : null;
4350
+ }
4351
+ function rememberMissing(p, reason) {
4352
+ const m = { checked: today(), reason };
4353
+ mkdirSync9(p.entryDir, { recursive: true });
4354
+ writeAtomic(missingFile(p), `checked: ${m.checked}
4355
+ reason: ${m.reason}
4356
+ `);
4357
+ return m;
4358
+ }
4359
+ var missingNote = (m, retryDays) => `${m.reason} (checked ${m.checked}; asked again after ${retryDays} day${retryDays === 1 ? "" : "s"})`;
4360
+
4361
+ // src/pubmedbatch.ts
4362
+ var MAX_PMIDS = 200;
4363
+ var COLUMNS3 = ["pmid", "title", "first_author", "year", "publication_types", "retracted", "level"];
4364
+ var EFETCH2 = "https://eutils.ncbi.nlm.nih.gov/entrez/eutils/efetch.fcgi";
4365
+ function efetchBatchUrl(pmids) {
4366
+ const u = new URL(EFETCH2);
4367
+ u.searchParams.set("db", "pubmed");
4368
+ u.searchParams.set("id", pmids.join(","));
4369
+ u.searchParams.set("retmode", "xml");
4370
+ if (process.env.FH_LIBRARY_NCBI_API_KEY) u.searchParams.set("api_key", process.env.FH_LIBRARY_NCBI_API_KEY);
4371
+ u.searchParams.set("tool", "fh-library");
4372
+ u.searchParams.set("email", contactEmail());
4373
+ return u.toString();
4374
+ }
4375
+ function splitArticles(xml) {
4376
+ const out = /* @__PURE__ */ new Map();
4377
+ for (const m of xml.matchAll(/<(PubmedArticle|PubmedBookArticle)\b[\s\S]*?<\/\1>/g)) {
4378
+ const pmid = /<PMID[^>]*>(\d+)<\/PMID>/.exec(m[0])?.[1];
4379
+ if (pmid && !out.has(pmid)) out.set(pmid, `<?xml version="1.0" ?>
4380
+ <PubmedArticleSet>${m[0]}</PubmedArticleSet>
4381
+ `);
4382
+ }
4383
+ return out;
4384
+ }
4385
+ var row2 = (pmid, s, level) => s ? [pmid, s.title, s.first_author, s.year, s.publication_types.length ? s.publication_types.join("; ") : null, s.retracted, level] : [pmid, null, null, null, null, null, level];
4386
+ async function fetchArticles(ids, libraryDir = FH_LIBRARY_DIR, converter = "fh-library", fetchImpl = fetchText) {
4387
+ const result = { columns: COLUMNS3, rows: [], stored: 0, cached: 0, not_found: 0, invalid: 0 };
4388
+ const order2 = [];
4389
+ const invalid = /* @__PURE__ */ new Map();
4390
+ for (const raw of ids.slice(0, MAX_PMIDS)) {
4391
+ const p = parseId(String(raw));
4392
+ if (p.error || p.type !== "pmid" || !/^\d{1,8}$/.test(p.id)) {
4393
+ const k = String(raw).trim();
4394
+ if (!invalid.has(k)) {
4395
+ invalid.set(k, "invalid");
4396
+ order2.push(k);
4397
+ }
4398
+ continue;
4399
+ }
4400
+ if (!order2.includes(p.id)) order2.push(p.id);
4401
+ }
4402
+ const status = (pmid) => {
4403
+ const paths = entryPaths(libraryDir, "pmid", pmid);
4404
+ return entryExists(paths) ? readStatus(paths, emptyStatus(`pmid:${pmid}`, "pmid", pmid, libraryDir)) : null;
4405
+ };
4406
+ const retry = settings().retry_after_days;
4407
+ const known = (pmid) => !!freshMissing(entryPaths(libraryDir, "pmid", pmid), retry);
4408
+ const missing = order2.filter((k) => !invalid.has(k) && !status(k) && !known(k));
4409
+ let records = /* @__PURE__ */ new Map();
4410
+ if (missing.length) records = splitArticles(await fetchImpl(efetchBatchUrl(missing)));
4411
+ for (const k of order2) {
4412
+ if (invalid.has(k)) {
4413
+ result.rows.push(row2(k, null, "invalid"));
4414
+ result.invalid++;
4415
+ continue;
4416
+ }
4417
+ let s = status(k);
4418
+ if (s) {
4419
+ result.rows.push(row2(k, s, String(s.level)));
4420
+ result.cached++;
4421
+ continue;
4422
+ }
4423
+ const paths = entryPaths(libraryDir, "pmid", k);
4424
+ if (known(k)) {
4425
+ result.rows.push(row2(k, null, "not found"));
4426
+ result.not_found++;
4427
+ continue;
4428
+ }
4429
+ const xml = records.get(k);
4430
+ const record = xml ? parsePubmedXml(xml) : null;
4431
+ if (!record) {
4432
+ rememberMissing(paths, "pmid not found");
4433
+ result.rows.push(row2(k, null, "not found"));
4434
+ result.not_found++;
4435
+ continue;
4436
+ }
4437
+ if (!entryExists(paths)) writeEntry(paths, record, xml, converter);
4438
+ s = status(k);
4439
+ result.rows.push(row2(k, s, String(s.level)));
4440
+ result.stored++;
4441
+ }
4442
+ return result;
4443
+ }
4444
+
4212
4445
  // src/tools.ts
4213
4446
  var pkg = (() => {
4214
4447
  try {
4215
- return JSON.parse(readFileSync11(new URL("../package.json", import.meta.url), "utf8"));
4448
+ return JSON.parse(readFileSync13(new URL("../package.json", import.meta.url), "utf8"));
4216
4449
  } catch {
4217
4450
  return { name: "fh-library", version: "0.0.0" };
4218
4451
  }
@@ -4232,6 +4465,8 @@ Tools
4232
4465
  - library_status(id) \u2014 the status of one record without downloading; level "missing" when it is not stored.
4233
4466
  - library_read(id, section?, query?, max_chars?, max_hits?) \u2014 one small piece of a stored document: its outline, one section, or the passages matching a query.
4234
4467
  - library_verify(claim, id) \u2014 does this one source state this one finding? A decision model judges the stored text and returns a verdict.
4468
+ - library_pubmed_fetch_articles(pmids) \u2014 title, first author, year, publication types, retracted and level for up to 200 PMIDs at once; faster and far smaller than asking PubMed, known papers from disk, the rest in one request and stored. For abstract or full text: library_read.
4469
+ - library_pubmed_search_articles(query, max_results?, sort?, years?) \u2014 search PubMed in its own syntax: the total and rows in columns order (PMID, title, first author, year, journal, publication types, PMCID, library level); faster and far smaller than asking PubMed. Cached a week.
4235
4470
  - library_clinicaltrials_search_studies(query | intervention | condition, status?, sort?, max_results?) \u2014 search ClinicalTrials.gov: the total and rows in columns order (NCT ID, title, status, phase, enrollment, dates, primary outcome, conditions, interventions, has_results). Cached a week.
4236
4471
  - library_clinicaltrials_get_study_record(nct) \u2014 one trial as a review needs it (status, design, enrollment, dates, outcomes, eligibility, result publications, summary), from the stored record; added first when missing.
4237
4472
  - library_clinicaltrials_get_study_results(nct, max_chars?) \u2014 a trial's posted results as Markdown, capped; the outcome titles listed for library_read.
@@ -4241,7 +4476,7 @@ Hints
4241
4476
  - library_read and library_verify store a missing record themselves; the library_read answer then carries added: true. No library_add necessary first.
4242
4477
  - citable false \u2014 the paper is retracted (a retraction notice, or PubMed's publication type), or holds nothing to read: no abstract and no full text. comment says which; corrections lists the notices as {type, pmid, source}.
4243
4478
  - level \u2014 what is stored: "full_text", "abstract" (no full text could be found; attempts names each source tried and why it failed) or "title" (not even an abstract; citable false).
4244
- - error "pmid not found" or "invalid id" \u2014 the ID itself is wrong; the same call gives the same answer.
4479
+ - error "pmid not found", "doi not found", "ctgov: not a study record", "dailymed: not an SPL document" \u2014 no service knows the ID. Remembered for a day (FH_LIBRARY_RETRY_AFTER_DAYS): the same call answers from disk with the date it was checked; after that the service is asked again. "invalid id" \u2014 the ID itself is malformed; the same call gives the same answer.
4245
4480
  - Calls may run in parallel: library_add, library_read and library_verify. Two calls on the same missing record share one add.
4246
4481
  - library_fetch takes any URL that is not a paper, label or trial record; those have IDs, and library_add, library_read and library_verify work on the stored document. A second pass over the same review reads every page from disk; a failed fetch is remembered for a day.
4247
4482
  - To verify that a web page exists and what it is: library_fetch(url, title_only: true). The answer is "Contents of <url>:" and the page's title line \u2014 a few tokens \u2014 or the fetch error; the page itself is stored in full, so a later library_fetch of the same URL reads from disk.
@@ -4337,7 +4572,8 @@ function registerTools(server2) {
4337
4572
  const xml = await fetchSpl(setid);
4338
4573
  const s = parseSpl(xml);
4339
4574
  if (!s) {
4340
- log("library_add", requested, "dailymed failed: not an SPL document");
4575
+ rememberMissing(paths, "dailymed: not an SPL document");
4576
+ log("library_add", requested, "dailymed failed: not an SPL document (remembered)");
4341
4577
  return { ...status, error: "dailymed: not an SPL document" };
4342
4578
  }
4343
4579
  const meta = writeLabelEntry(paths, s, xml, `${pkg.name} ${pkg.version}`);
@@ -4345,7 +4581,9 @@ function registerTools(server2) {
4345
4581
  return statusFromMeta(paths, meta, status);
4346
4582
  } catch (e) {
4347
4583
  const msg = e instanceof Error ? e.message : String(e);
4348
- log("library_add", requested, `dailymed failed: ${msg}`);
4584
+ const gone = /not an SPL document|^HTTP 404\b/.test(msg);
4585
+ if (gone) rememberMissing(paths, msg.startsWith("HTTP 404") ? "dailymed: no label for this set ID (HTTP 404)" : msg);
4586
+ log("library_add", requested, `dailymed failed: ${msg}${gone ? " (remembered)" : ""}`);
4349
4587
  return { ...status, error: msg };
4350
4588
  }
4351
4589
  }
@@ -4378,7 +4616,8 @@ function registerTools(server2) {
4378
4616
  const json2 = await fetchStudy(nctid);
4379
4617
  const s = parseStudy(json2);
4380
4618
  if (!s) {
4381
- log("library_add", requested, "ctgov failed: not a study record");
4619
+ rememberMissing(paths, "ctgov: not a study record");
4620
+ log("library_add", requested, "ctgov failed: not a study record (remembered)");
4382
4621
  return { ...status, error: "ctgov: not a study record" };
4383
4622
  }
4384
4623
  const meta = writeStudyEntry(paths, s, json2, `${pkg.name} ${pkg.version}`);
@@ -4386,7 +4625,9 @@ function registerTools(server2) {
4386
4625
  return statusFromMeta(paths, meta, status);
4387
4626
  } catch (e) {
4388
4627
  const msg = e instanceof Error ? e.message : String(e);
4389
- log("library_add", requested, `ctgov failed: ${msg}`);
4628
+ const gone = /not a study record|^HTTP 404\b/.test(msg);
4629
+ if (gone) rememberMissing(paths, msg.startsWith("HTTP 404") ? "ctgov: no study for this NCT ID (HTTP 404)" : msg);
4630
+ log("library_add", requested, `ctgov failed: ${msg}${gone ? " (remembered)" : ""}`);
4390
4631
  return { ...status, error: msg };
4391
4632
  }
4392
4633
  }
@@ -4409,6 +4650,14 @@ function registerTools(server2) {
4409
4650
  async function addEntryNow(id) {
4410
4651
  const t0 = Date.now();
4411
4652
  let parsed = parseId(id);
4653
+ if (!parsed.error) {
4654
+ const mp = entryPaths(FH_LIBRARY_DIR, parsed.type, parsed.id);
4655
+ const miss2 = entryExists(mp) ? null : freshMissing(mp, settings().retry_after_days);
4656
+ if (miss2) {
4657
+ log("library_add", parsed.requested, `missing cached: ${miss2.reason} (${miss2.checked})`);
4658
+ return json({ ...emptyStatus(parsed.requested, parsed.type, parsed.id, FH_LIBRARY_DIR), entry: mp.entry, level: "missing", error: missingNote(miss2, settings().retry_after_days) });
4659
+ }
4660
+ }
4412
4661
  if (parsed.error) {
4413
4662
  log("library_add", parsed.requested, `error ${parsed.error}`);
4414
4663
  return json({ ...emptyStatus(parsed.requested, parsed.type, parsed.id, FH_LIBRARY_DIR), error: parsed.error });
@@ -4503,7 +4752,8 @@ function registerTools(server2) {
4503
4752
  }
4504
4753
  }
4505
4754
  if (!record) {
4506
- log("library_add", parsed.requested, "crossref failed: doi not found at Crossref or doi.org");
4755
+ rememberMissing(paths, "doi not found");
4756
+ log("library_add", parsed.requested, "crossref failed: doi not found at Crossref or doi.org (remembered)");
4507
4757
  return json({ ...status, error: "doi not found" });
4508
4758
  }
4509
4759
  if (!record.doi) record.doi = parsed.id;
@@ -4528,7 +4778,8 @@ function registerTools(server2) {
4528
4778
  }
4529
4779
  const record = parsePubmedXml(xml);
4530
4780
  if (!record) {
4531
- log("library_add", parsed.requested, "pubmed failed: pmid not found");
4781
+ rememberMissing(paths, "pmid not found");
4782
+ log("library_add", parsed.requested, "pubmed failed: pmid not found (remembered)");
4532
4783
  return json({ ...status, error: "pmid not found" });
4533
4784
  }
4534
4785
  meta = writeEntry(paths, record, xml, `${pkg.name} ${pkg.version}`);
@@ -4557,8 +4808,9 @@ function registerTools(server2) {
4557
4808
  log("library_status", parsed.requested, `level=${s.level}`);
4558
4809
  return json(s);
4559
4810
  }
4560
- log("library_status", parsed.requested, "level=missing");
4561
- return json({ ...status, entry: paths.entry, entry_dir: paths.entryDir, level: "missing" });
4811
+ const miss2 = freshMissing(paths, settings().retry_after_days);
4812
+ log("library_status", parsed.requested, miss2 ? `level=missing (${miss2.reason}, ${miss2.checked})` : "level=missing");
4813
+ return json({ ...status, entry: paths.entry, level: "missing", ...miss2 ? { error: missingNote(miss2, settings().retry_after_days) } : {} });
4562
4814
  }
4563
4815
  );
4564
4816
  server2.tool(
@@ -4589,8 +4841,8 @@ function registerTools(server2) {
4589
4841
  }
4590
4842
  added = true;
4591
4843
  }
4592
- const full = existsSync11(paths.fulltextMd);
4593
- const md = readFileSync11(full ? paths.fulltextMd : paths.md, "utf8");
4844
+ const full = existsSync13(paths.fulltextMd);
4845
+ const md = readFileSync13(full ? paths.fulltextMd : paths.md, "utf8");
4594
4846
  const text3 = full ? "full_text" : "abstract";
4595
4847
  const base = added ? { requested: parsed.requested, entry: paths.entry, text: text3, added: true } : { requested: parsed.requested, entry: paths.entry, text: text3 };
4596
4848
  const headings2 = () => outline(md, text3).sections;
@@ -4610,8 +4862,8 @@ function registerTools(server2) {
4610
4862
  const cap = max_chars ?? 6e3;
4611
4863
  let sec = readSection(md, section2, cap);
4612
4864
  let via = "";
4613
- if (!sec && /abstract/i.test(section2) && full && existsSync11(paths.md)) {
4614
- sec = readSection(readFileSync11(paths.md, "utf8"), "abstract", cap);
4865
+ if (!sec && /abstract/i.test(section2) && full && existsSync13(paths.md)) {
4866
+ sec = readSection(readFileSync13(paths.md, "utf8"), "abstract", cap);
4615
4867
  via = " (from the abstract file)";
4616
4868
  }
4617
4869
  if (!sec && /declaration/i.test(section2)) {
@@ -4680,7 +4932,7 @@ function registerTools(server2) {
4680
4932
  }
4681
4933
  log("library_info", null, `entries=${info.entries}${info.verify ? ` verify ${info.verify.model} ${info.verify.ok ? `ok ${info.verify.ms} ms` : `down: ${info.verify.error}`}` : " verify not configured"}`);
4682
4934
  const s = currentStats();
4683
- const cache = s ? { since: s.started, verify: s.verify, fetch: s.fetch, search: s.search } : null;
4935
+ const cache = s ? { since: s.started, verify: s.verify, fetch: s.fetch, search: s.search, pubmed: s.pubmed } : null;
4684
4936
  return { content: [{ type: "text", text: compact2({ ...info, cache }) }] };
4685
4937
  }
4686
4938
  );
@@ -4711,9 +4963,46 @@ function registerTools(server2) {
4711
4963
  return { content: [{ type: "text", text: r.error }], isError: true };
4712
4964
  }
4713
4965
  );
4966
+ server2.tool(
4967
+ "library_pubmed_fetch_articles",
4968
+ `Title, first author, year, publication types, retracted and library level for up to ${MAX_PMIDS} PubMed IDs in one call. Faster and far smaller than asking PubMed: known papers come from disk, the rest in one request, stored for every later call; rows as arrays in columns order, ~120 characters a paper. Level "not found": PubMed has no such record. For abstract or full text: library_read.`,
4969
+ { pmids: z.array(z.string()).min(1).max(MAX_PMIDS).describe('PubMed IDs, "30955454" or "pmid:30955454", up to 200.') },
4970
+ async ({ pmids }) => {
4971
+ const t0 = Date.now();
4972
+ const r = await fetchArticles(pmids, FH_LIBRARY_DIR, `${pkg.name} ${pkg.version}`);
4973
+ if (r.cached) bump("pubmed", true, r.cached);
4974
+ if (r.stored) bump("pubmed", false, r.stored);
4975
+ const secs = ((Date.now() - t0) / 1e3).toFixed(1).padStart(4, "0") + "s";
4976
+ log("library_pubmed", `pmids:${r.rows.length}`, `fetch_articles ${secs} cached=${r.cached} stored=${r.stored}${r.not_found ? ` not_found=${r.not_found}` : ""}${r.invalid ? ` invalid=${r.invalid}` : ""}`);
4977
+ return { content: [{ type: "text", text: leanJson(r) }] };
4978
+ }
4979
+ );
4980
+ server2.tool(
4981
+ "library_pubmed_search_articles",
4982
+ `Search PubMed with its own query syntax (AND / OR, field tags like [pt], [ti], [mesh]); sort relevance | newest; years "2020-2026". Faster and far smaller than asking PubMed: the total and up to ${MAX_RESULTS2} rows as arrays in columns order \u2014 PMID, title, first author, year, journal, publication types, PMCID when free full text exists, and the library level when the paper is already stored. The same search is answered from disk for ${searchCacheDays()} days. For a batch of PMIDs: library_pubmed_fetch_articles; for abstract or full text: library_read.`,
4983
+ {
4984
+ query: z.string().describe('PubMed query, full syntax: "tadalafil AND (systematic review[pt] OR meta-analysis[pt])".'),
4985
+ max_results: z.number().int().min(1).max(MAX_RESULTS2).default(20).describe("Rows to return (1\u2013100)."),
4986
+ sort: z.enum(["relevance", "newest"]).default("relevance").describe("relevance (PubMed's ranking) or newest (publication date, latest first)."),
4987
+ years: z.string().optional().describe('Publication years: "2020-2026", "2022-", "-2019" or "2022".')
4988
+ },
4989
+ async (args) => {
4990
+ const t0 = Date.now();
4991
+ const r = await searchArticles(args, FH_LIBRARY_DIR);
4992
+ const secs = ((Date.now() - t0) / 1e3).toFixed(1).padStart(4, "0") + "s";
4993
+ if (!r.ok) {
4994
+ log("library_search", "pm:-", `error: ${r.error}`);
4995
+ return { content: [{ type: "text", text: compact2({ error: r.error }) }], isError: true };
4996
+ }
4997
+ bump("search", r.cached);
4998
+ const q = r.result.query;
4999
+ log("library_search", `pm:${logId2(q)}`, `${r.cached ? "cached" : `searched ${secs}`} rows=${r.result.rows.length} total=${r.result.total}${q.sort !== "relevance" ? ` sort=${q.sort}` : ""}${q.years ? ` years=${q.years}` : ""} | ${q.query}`);
5000
+ return { content: [{ type: "text", text: leanJson(r.result) }] };
5001
+ }
5002
+ );
4714
5003
  server2.tool(
4715
5004
  "library_clinicaltrials_search_studies",
4716
- `Search ClinicalTrials.gov by free text, intervention or condition; status ongoing | completed | all; sort relevance | newest | largest | completion. Returns the total and up to ${MAX_RESULTS} rows as arrays in columns order: NCT ID, title, status, phase, enrollment, start, primary completion, primary outcome, conditions, interventions, has_results. Link: https://clinicaltrials.gov/study/<NCT ID>. Cached ${SEARCH_AFTER_DAYS} days. One trial in detail: library_clinicaltrials_get_study_record.`,
5005
+ `Search ClinicalTrials.gov by free text, intervention or condition; status ongoing | completed | all; sort relevance | newest | largest | completion. Returns the total and up to ${MAX_RESULTS} rows as arrays in columns order: NCT ID, title, status, phase, enrollment, start, primary completion, primary outcome, conditions, interventions, has_results. Link: https://clinicaltrials.gov/study/<NCT ID>. Cached ${searchCacheDays()} days. One trial in detail: library_clinicaltrials_get_study_record.`,
4717
5006
  {
4718
5007
  query: z.string().optional().describe("Free-text search across all fields (title, conditions, interventions, summary, \u2026); AND / OR / NOT allowed."),
4719
5008
  intervention: z.string().optional().describe('Intervention or treatment, e.g. "tadalafil", "cognitive behavioral therapy".'),
@@ -4760,7 +5049,7 @@ function registerTools(server2) {
4760
5049
  return { content: [{ type: "text", text: compact2({ error: e.error }) }], isError: true };
4761
5050
  }
4762
5051
  const raw = e.paths.fulltextRaw("json");
4763
- const r = existsSync11(raw) ? recordSummary(readFileSync11(raw, "utf8")) : null;
5052
+ const r = existsSync13(raw) ? recordSummary(readFileSync13(raw, "utf8")) : null;
4764
5053
  if (!r) {
4765
5054
  log("library_trials", e.requested, "record error: no stored record");
4766
5055
  return { content: [{ type: "text", text: compact2({ error: "no stored record" }) }], isError: true };
@@ -4779,7 +5068,7 @@ function registerTools(server2) {
4779
5068
  log("library_trials", nct, `results error: ${e.error}`);
4780
5069
  return { content: [{ type: "text", text: compact2({ error: e.error }) }], isError: true };
4781
5070
  }
4782
- const md = existsSync11(e.paths.fulltextMd) ? readFileSync11(e.paths.fulltextMd, "utf8") : "";
5071
+ const md = existsSync13(e.paths.fulltextMd) ? readFileSync13(e.paths.fulltextMd, "utf8") : "";
4783
5072
  const r = resultsText(e.requested.replace(/^nct:/, "NCT"), md, max_chars);
4784
5073
  log("library_trials", e.requested, `results ${r.has_results ? `${r.outcomes.length} outcomes${r.truncated ? " truncated" : ""}` : "none"}`);
4785
5074
  return { content: [{ type: "text", text: leanJson(r) }] };
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "fh-library",
3
- "version": "0.2.11",
3
+ "version": "0.2.13",
4
4
  "mcpName": "io.github.forever-healthy/fh-library",
5
5
  "description": "MCP Server that builds a local Markdown library of the scientific papers cited by evidence reviews — fetches metadata, abstract and full text per PubMed ID through a configurable cascade of retrieval services",
6
6
  "type": "module",