mcp-libraryofcongress 1.1.0 → 2.0.0

package/CHANGELOG.md

@@ -1,5 +1,40 @@
 # Changelog
 
+## 2.0.0
+
+### Breaking change
+
+`search_newspapers` no longer returns `words_located`. Each match now carries
+`excerpt_kind`, one of `"passage"` or `"page_opening"`. A caller reading
+`words_located === true` reads `excerpt_kind === "passage"`, and the false case
+reads `"page_opening"`.
+
+The boolean and the excerpt disagreed about what they were. `words_located:
+false` means the machine-read text the Library returned stops before the
+searched words, so the excerpt beside it is the **opening of the page** and does
+not carry the match. It sat in the answer in the same place and the same shape
+as a real passage, and a note beside it was the only thing saying otherwise.
+Measured on a live search, four matches in five were openings. Naming the kind
+of thing an excerpt is puts that where it cannot be skipped.
+
+### Also in this release
+
+- Every excerpt is now labelled in the rendered text itself, as
+  `[passage]` or `[page opening]` ahead of the words, so the mark travels with
+  the text a reader copies.
+- The tool described quoting as matching a phrase whole. It does not. Measured:
+  `"hell is other people"` quoted returns 18 pages, unquoted 1,634,731, and
+  among the quoted results are pages from 1891, 1897 and 1906, which precede the
+  line by half a century. Quoting narrows hard and decides nothing about how the
+  Library matches. The promise is gone from the tool description and from both
+  halves of the README, and a quoted query now carries a note saying the match
+  is not guaranteed to be the phrase.
+- Headings said the pages carry the query. They say what the Library matched.
+
+`NewspaperHit.wordsLocated`, on the `./client` subpath, is unchanged: it is the
+raw fact about the text received, and the tool layer names the kind of thing it
+renders from it.
+
 ## 1.1.0
 
 - An argument no tool declared was read and dropped, and the answer came back

package/README.md

@@ -64,12 +64,15 @@ The server is **read-only**. It uploads nothing and writes nothing back.
 A catalogue search reads titles, creators and descriptions. `search_newspapers`
 reads what optical character recognition took off millions of scanned pages of
 American newspapers, so it answers a question nothing else here can: _which
-issue printed this phrase_. A match comes back with the paper, the date, the
+issue printed these words_. A match comes back with the paper, the date, the
 leaf of the issue, the state it was published in, and an address that opens that
 leaf with the query applied.
 
-Put a phrase in double quotes to match it whole. Without quotes the words are
-matched separately, which finds far more.
+Double quotes narrow the search sharply, and the Library decides what they mean:
+a matched page can carry the words apart or in another order rather than the
+phrase as written, so an answer to a quoted query says so and points at the page
+to read. Without quotes the words are matched separately, which finds far more
+again.
 
 The corpus spans every state and a century and a half, so a bare phrase reaches
 a great deal that a question did not ask for. Three arguments narrow it:
@@ -101,9 +104,14 @@ page 2, 3 and so on rather than treating the first answer as the whole of it.
 
 **An excerpt is sometimes the opening of the page.** The Library returns the
 beginning of a page's text with each row rather than the whole page, so the
-searched words are often further down than that text reaches. `words_located`
-says which of the two happened: true and the passage is centred on the words,
-false and it is the opening of the leaf. Follow `source_url` for the rest.
+searched words are often further down than that text reaches. Every match
+carries `excerpt_kind`, and every excerpt is labelled with it in the text block:
+
+- **`passage`**, the text around the words that matched, centred on them.
+- **`page_opening`**, the start of the leaf, sent because the text that came
+  back stops before the searched words appear. It does not carry the match, so
+  quoting it quotes something else. The notes count how many there are, and
+  `source_url` opens the leaf with the query applied.
 
 **Scanned text is machine-read.** Excerpts carry the misreadings that come with
 it. Quote them as scanned text and link the page.
@@ -338,13 +346,17 @@ Le serveur est **en lecture seule**. Il ne téléverse rien et n'écrit rien.
 Une recherche de catalogue lit les titres, les auteurs et les descriptions.
 `search_newspapers` lit ce que la reconnaissance de caractères a tiré de
 millions de pages de journaux américains numérisées, et répond donc à une
-question qu'aucun autre outil ici ne sait traiter : _quel numéro a imprimé cette
-phrase_. Une correspondance revient avec le journal, la date, le feuillet du
+question qu'aucun autre outil ici ne sait traiter : _quel numéro a imprimé ces
+mots_. Une correspondance revient avec le journal, la date, le feuillet du
 numéro, l'État de publication, et une adresse qui ouvre ce feuillet avec la
 requête appliquée.
 
-Mettez une phrase entre guillemets doubles pour la faire correspondre en entier.
-Sans guillemets, les mots sont cherchés séparément, ce qui trouve bien plus.
+Les guillemets doubles resserrent fortement la recherche, et c'est la Library
+qui décide de ce qu'ils veulent dire : une page retenue peut porter les mots
+éloignés les uns des autres ou dans un autre ordre plutôt que la phrase telle
+qu'elle est écrite. La réponse à une requête entre guillemets le dit et renvoie
+à la page à lire. Sans guillemets, les mots sont cherchés séparément, ce qui
+trouve bien plus encore.
 
 Le corpus couvre tous les États et un siècle et demi : une phrase seule ramène
 donc beaucoup de choses que la question ne demandait pas. Trois arguments la
@@ -379,9 +391,15 @@ que de prendre la première réponse pour la totalité.
 **Un extrait est parfois le début de la page.** La Library renvoie le début du
 texte d'une page avec chaque ligne de résultat plutôt que la page entière, si
 bien que les mots cherchés se trouvent souvent plus bas que ce texte ne va.
-`words_located` indique lequel des deux cas s'est produit : vrai et le passage
-est centré sur les mots, faux et c'est le début du feuillet. Suivez `source_url`
-pour la suite.
+Chaque correspondance porte `excerpt_kind`, et chaque extrait en porte
+l'étiquette dans le bloc de texte :
+
+- **`passage`** : le texte autour des mots correspondants, centré sur eux.
+- **`page_opening`** : le début du feuillet, renvoyé parce que le texte reçu
+  s'arrête avant que les mots cherchés n'apparaissent. Il ne porte pas la
+  correspondance, donc le citer revient à citer autre chose. Les notes comptent
+  combien il y en a, et `source_url` ouvre le feuillet avec la requête
+  appliquée.
 
 **Le texte numérisé est lu par une machine.** Les extraits en portent les
 fautes. Citez-les comme tels et suivez le lien.

package/dist/index.js

@@ -7,7 +7,7 @@ import { StdioServerTransport } from "@modelcontextprotocol/sdk/server/stdio.js"
 import { McpServer } from "@modelcontextprotocol/sdk/server/mcp.js";
 
 // src/version.ts
-var PKG_VERSION = "1.1.0";
+var PKG_VERSION = "2.0.0";
 var REPO_URL = "https://github.com/smeet666/mcp-libraryofcongress";
 
 // src/config.ts
@@ -1489,11 +1489,11 @@ import { z as z6 } from "zod";
 var searchNewspapersDescription = [
   "Search the text inside digitised American newspaper pages held by the Library of Congress.",
   "This reads what optical recognition took off the scanned pages, so it finds a phrase that appears nowhere in a title or a catalogue record.",
-  "Put a phrase in double quotes to match it whole; without quotes the words are matched separately, which finds far more.",
+  "Double quotes narrow the search sharply, and the Library decides how it matches what is inside them: a page can come back carrying the words apart or in another order rather than the phrase as written. Without quotes the words are matched separately, which finds far more again.",
   "'total' counts the pages that match, and they page: ask for page 2, 3 and so on to see beyond the first answer. It is not a count of how many times the words occur.",
   "Each match names the newspaper, the date, the leaf of the issue and the state it was published in, and 'source_url' opens that leaf with the query applied.",
   "'location' keeps to papers published in one state, 'publication' to a single paper, and 'year_from' with 'year_to' to a span of years. A filter matching nothing is dropped and the answer says so.",
-  "The Library returns the opening of a page's text with each row rather than the whole page, so the searched words are often further down than the excerpts reach: 'words_located' says which of the two happened for each match.",
+  "Every match carries 'excerpt_kind', and the excerpts are labelled with it in the text. A 'passage' is the text around the words that matched. A 'page_opening' is the start of the page, sent because the text the Library returned with the row stops before those words appear, so it does not carry the match and quoting it quotes something else.",
   "Use search_items instead when looking for a work by its title, creator or subject."
 ].join(" ");
 var searchNewspapersInput = strictInput({
@@ -1531,15 +1531,16 @@ var searchNewspapersOutput = z6.object({
       published_on: z6.string().nullable().describe("Date of the issue, as published."),
       publication: z6.string().nullable().describe("The newspaper, with the years it ran."),
       state: z6.string().nullable().describe("Where the paper was published."),
-      words_located: z6.boolean().describe(
-        "True when the searched words were found in the text returned with this row, in which case the excerpts are centred on them. False when they sit further down the page than that text reaches, in which case the excerpts are its opening."
+      excerpt_kind: z6.enum(["passage", "page_opening"]).describe(
+        "'passage' means the excerpts are the text around the words that matched, centred on them because they were found in the text returned with this row. 'page_opening' means they are the start of the page, sent because that text stops before the searched words appear, so a page_opening does not carry the match."
       ),
-      excerpts: z6.array(z6.string()).describe("Passages as a machine read them off the page."),
+      excerpts: z6.array(z6.string()).describe("Machine-read text off the page, all of the kind 'excerpt_kind' names."),
       source_url: z6.string().describe("Opens the leaf itself, with the query applied.")
     })
   ),
   notes: z6.array(z6.string())
 });
+var QUOTED_PHRASE = /"[^"]+"/;
 function describeNarrowing2(args) {
   const written = [];
   if (args.location) written.push(`location="${args.location}"`);
@@ -1578,7 +1579,7 @@ async function runSearchNewspapers(client, args) {
       const wider = await client.searchNewspapers(args.query, args.limit, args.page, budget);
       if (wider.data.paging.resultCount > 0) {
         notes.push(
-          `No page carrying ${args.query} matches ${narrowing.join(", ")}. Those were set aside and the search was asked again without them, so the matches below are unfiltered. The corpus uses its own wording: read 'state' and 'publication' on a match to see the form it expects.`
+          `The Library matched no page for ${args.query} under ${narrowing.join(", ")}. Those were set aside and the search was asked again without them, so the matches below are unfiltered. The corpus uses its own wording: read 'state' and 'publication' on a match to see the form it expects.`
         );
         result = wider;
       }
@@ -1592,6 +1593,7 @@ async function runSearchNewspapers(client, args) {
     }
     const hits = data.hits.map((hit) => ({
       identifier: hit.identifier,
+      excerpt_kind: hit.wordsLocated ? "passage" : "page_opening",
       title: hit.title,
       creator: hit.creator,
       year: hit.year,
@@ -1599,16 +1601,22 @@ async function runSearchNewspapers(client, args) {
       published_on: hit.publishedOn,
       publication: hit.publication,
       state: hit.state,
-      words_located: hit.wordsLocated,
       excerpts: hit.excerpts.map((excerpt) => truncate(excerpt, args.max_excerpt_chars + 2)),
       source_url: hit.sourceUrl
     }));
     const total = data.paging.resultCount;
     if (hits.length > 0) notes.push(OCR_CAVEAT);
-    const elsewhere = hits.filter((hit) => !hit.words_located).length;
-    if (elsewhere > 0) {
+    const openings = hits.filter(
+      (hit) => hit.excerpt_kind === "page_opening" && hit.excerpts.length > 0
+    ).length;
+    if (openings > 0) {
       notes.push(
-        `On ${elsewhere} of ${hits.length} match(es) the searched words sit further down the page than the text returned with the row, so those excerpts are the opening of the page rather than the passage that matched. Follow source_url, which opens the leaf with the query applied.`
+        `On ${openings} of ${hits.length} match(es) the searched words sit further down the page than the text returned with the row, so those excerpts are the opening of the page rather than the passage that matched, and each is labelled [page opening]. Quoting one of them quotes something else: follow source_url, which opens the leaf with the query applied.`
+      );
+    }
+    if (hits.length > 0 && QUOTED_PHRASE.test(args.query)) {
+      notes.push(
+        "The query carries double quotes. That narrows the search sharply, and the Library decides what the quotes mean: it does not guarantee that a matched page carries the words together in that order, so a match can be a page where they sit apart. Read the page behind source_url before repeating the query as the phrase it printed."
       );
     }
     if (total > hits.length) {
@@ -1618,7 +1626,7 @@ async function runSearchNewspapers(client, args) {
     }
     if (total === 0) {
       notes.push(
-        "No digitised newspaper page carries these words. An unquoted query matches the words separately, which usually finds more."
+        "The Library matched no digitised newspaper page for these words. An unquoted query matches the words separately, which usually finds more."
       );
     }
     if (hits.length === 0 && total > 0) {
@@ -1626,7 +1634,7 @@ async function runSearchNewspapers(client, args) {
         `Page ${args.page} is past the last match. ${total} pages match, so ask for a lower page.`
       );
     }
-    const body = hits.length === 0 ? total > 0 ? `Page ${args.page} is past the last of ${total} newspaper pages carrying ${args.query}.` : `Nothing found in the scanned newspapers for ${args.query}.` : `${hits.length} of ${total} newspaper pages carrying ${args.query}:
+    const body = hits.length === 0 ? total > 0 ? `Page ${args.page} is past the last of ${total} newspaper pages the Library matched for ${args.query}.` : `Nothing found in the scanned newspapers for ${args.query}.` : `${hits.length} of ${total} newspaper pages the Library matched for ${args.query}:
 ` + hits.map((hit, index) => {
       const where = [
         `${index + 1}. ${hit.publication ?? hit.title ?? "untitled"}`,
@@ -1634,7 +1642,8 @@ async function runSearchNewspapers(client, args) {
         hit.page_number === null ? "" : `\xB7 page ${hit.page_number}`,
         hit.state ? `\xB7 ${hit.state}` : ""
       ].filter(Boolean).join(" ");
-      const passages = hit.excerpts.map((excerpt) => `     ${excerpt}`).join("\n");
+      const label = hit.excerpt_kind === "page_opening" ? "[page opening]" : "[passage]";
+      const passages = hit.excerpts.map((excerpt) => `     ${label} ${excerpt}`).join("\n");
       return `${where}
 ${passages}
      ${hit.source_url}`;
@@ -1655,11 +1664,13 @@ var READ_ONLY = {
 var INSTRUCTIONS = [
   "Tools for the Library of Congress. No API key and no account are needed.",
   "Two questions lead to two different tools, and choosing the wrong one gives a confident empty answer.",
-  "To find a phrase printed on a page of an American newspaper, use search_newspapers: it reads the text scanned off the pages and returns the paper, the date, the leaf and the passage.",
+  "To find words printed on a page of an American newspaper, use search_newspapers: it reads the text scanned off the pages and returns the paper, the date, the leaf and what it holds of that text.",
   "To find a work by its title, creator or subject, use search_items, which requires a 'media_type' because the Library keeps a separate catalogue per kind of thing.",
   "list_collections shows the corpora a curator built, and each row carries the wording search_items takes as its 'collection' filter.",
   "search_newspapers reports 'total' as the number of pages that match, and they page: ask for the next page rather than treating the first answer as the whole of it.",
-  "Excerpts come from optical character recognition, so the words can be wrong; repeat them as scanned text and link the page. When 'words_located' is false the searched words sit further down the page than the text this server received, and the excerpt is the opening of that page.",
+  "Excerpts come from optical character recognition, so the words can be wrong; repeat them as scanned text and link the page.",
+  "Every match carries 'excerpt_kind', and the excerpts are labelled with it in the text. A 'passage' is the text around the words that matched. A 'page_opening' is the start of the page, sent because the text this server received stops before those words appear, so it does not carry the match and citing it as the match cites the wrong words. The notes count the openings.",
+  "Double quotes narrow a newspaper search sharply, and the Library decides what they mean: a matched page can carry the words apart or in another order rather than the phrase as written, so read the page behind source_url before saying it printed the phrase.",
   "Metadata here is in the public domain, and individual items are not: a record states its own terms, and a record stating none is not a record granting permission.",
   "This server paces itself to the ceiling the Library publishes, so answers take a few seconds. A rate_limited error means it was asked to slow down, never that the thing you asked for is missing.",
   "Every result carries a source_url. Credit the Library of Congress and link what you use."