@fbraza/pi-cite 0.2.0 → 0.3.1

package/README.md

@@ -1,24 +1,22 @@
 # @fbraza/pi-cite
 
 A standalone [Pi](https://pi.dev) extension providing literature-research tools for
-academic workflows. Registers four tools callable by the agent:
+academic workflows. Registers two tools callable by the agent:
 
-- **`literature_search`** — PubMed-first search with optional Semantic Scholar
-  supplementary metadata.
+- **`literature_search`** — literature workflow search against PubMed using a
+  PubMed-ready query (MeSH `[mh]`, `[tiab]`, `[pt]`, substance `[nm]`, and Boolean
+  logic), with streaming progress and deduplicated results.
 - **`pubmed_search`** — direct PubMed query (MeSH, `[tiab]`, `[pt]`, etc.).
-- **`fetch_fulltext`** — retrieve a paper PDF via PMC → publisher OA → fallback.
-- (`semantic_scholar` helper used internally by the search tools.)
 
 ## Bundled skill
 
 Ships with the **`literature`** skill (`skills/literature/`), which turns these
-tools into an end-to-end review workflow: verified-citation search, full-text
-retrieval, per-paper experiment extraction, and a structured hypothesis
-synthesis. Its frontmatter declares `allowed-tools` covering the extension's
-tools above, so the skill and extension are paired on purpose.
+tools into an end-to-end review workflow: verified-citation search, per-paper
+experiment extraction, and a structured hypothesis synthesis. Its frontmatter
+declares `allowed-tools` covering the extension's tools above, so the skill and
+extension are paired on purpose.
 
-- `references/` — PubMed/Semantic Scholar query syntax, API reference, and
-  full-text access routines.
+- `references/` — PubMed query syntax, API reference, and common queries.
 - `scripts/` — Python helpers (`extract_experiments.py`, `synthesis.py`,
   `generate_table.py`, `export_all.py`) invoked by the skill.
 
@@ -54,4 +52,3 @@ npm run pack:check  # preview the published tarball contents
 | Variable | Purpose |
 |---|---|
 | `NCBI_API_KEY` / `api_key` env | PubMed rate limit + E-utilities auth |
-| `SEMANTIC_SCHOLAR_API_KEY` | Enables Semantic Scholar supplementary search |

package/package.json

@@ -1,7 +1,7 @@
 {
   "name": "@fbraza/pi-cite",
-  "version": "0.2.0",
-  "description": "Pi extension with PubMed, Semantic Scholar, literature search, and full-text retrieval tools.",
+  "version": "0.3.1",
+  "description": "Pi extension with PubMed and literature search tools.",
   "license": "MIT",
   "type": "module",
   "files": [
@@ -13,8 +13,7 @@
     "pi-package",
     "pi-extension",
     "literature",
-    "pubmed",
-    "semantic-scholar"
+    "pubmed"
   ],
   "pi": {
     "extensions": [

package/skills/literature/SKILL.md

@@ -1,7 +1,7 @@
 ---
 name: literature
-description: Unified literature search, verification, full-text retrieval, and synthesis workflow for scientific questions. Use when any biological claim needs a verified citation, when reviewing a gene/pathway/disease/drug/target, when surveying preclinical evidence for a target in a disease, when checking novelty, when retrieving full text for specific papers, or when turning a paper set into a structured hypothesis synthesis.
-allowed-tools: Read, Write, WebFetch, WebSearch, literature_search, pubmed_search, semantic_scholar_search, fetch_fulltext
+description: Unified literature search, verification, and synthesis workflow for scientific questions. Use when any biological claim needs a verified citation, when reviewing a gene/pathway/disease/drug/target, when surveying preclinical evidence for a target in a disease, when checking novelty, or when turning a paper set into a structured hypothesis synthesis.
+allowed-tools: Read, Write, WebFetch, WebSearch, literature_search, pubmed_search
 starting-prompt: Conduct a literature review on my research topic with verified citations, structured synthesis, and a per-paper summary table.
 ---
 
@@ -16,7 +16,6 @@ Use this skill when you need to:
 - review literature on a gene, pathway, disease, drug, or molecular target
 - survey preclinical evidence for a target in a disease context
 - check whether a finding appears novel or already published
-- retrieve full text or PDFs for key papers
 - synthesize a paper set into hypotheses, contradictions, and evidence-weighted conclusions
 
 Do not use this skill for:
@@ -30,7 +29,6 @@ Do not use this skill for:
 - Never fabricate PMIDs, DOIs, titles, journals, years, or author lists.
 - Distinguish human, animal, and in vitro evidence.
 - Weight evidence quality by study design and replication.
-- Record how full text was obtained for each paper.
 - Use inline numbered citations like `[1]` or `[1, 2]` in narrative synthesis.
 - Never overwrite outputs from a previous literature search.
 - Never write literature-review outputs directly to generic shared paths under `results/`.
@@ -48,26 +46,29 @@ Always clarify:
 
 ### Step 2 — Create a dedicated output folder
 
-For every new literature review or literature research task, create a new dedicated folder inside `results/` before generating files.
+For every new literature review or literature research task, create a new dedicated folder under `results/literature_review/` before generating files.
 
-The folder name must describe the search session/topic clearly, for example:
-- `results/literature_multiomics_ML_biomarkers_PGD/`
-- `results/literature_siRNA_lung_transplant_new_treatments/`
+Use the path `results/literature_review/<subject_of_study>/`, where `<subject_of_study>` is a short **snake_case title summary of the theme** of the literature search. Derive it from the scope clarified in Step 1: lower case, words separated by single underscores, no spaces, hyphens, or punctuation. For example, a review on **trained immunity in transplantation** becomes:
 
-All generated files for that search session must be saved inside this dedicated folder, including:
+- `results/literature_review/trained_immunity_in_transplantation/`
+
+Other examples:
+- `results/literature_review/sirna_lung_transplant_new_treatments/`
+- `results/literature_review/multiomics_ml_biomarkers_in_pgd/`
+
+All generated files for that search session must be saved inside this dedicated subject folder, including:
 - `literature_report.md`
 - `paper_summary_table.csv`
 - `search_log.md`
-- `pdfs/`
 - any optional analysis/export artifacts such as `analysis_object.pkl`
 
-Never write directly to generic shared paths such as:
+Never write outputs directly to the parent folder or to the `results/` root, for example:
+- `results/literature_review/literature_report.md`
+- `results/literature_review/paper_summary_table.csv`
+- `results/literature_review/analysis_object.pkl`
 - `results/literature_report.md`
-- `results/paper_summary_table.csv`
-- `results/analysis_object.pkl`
-- `results/literature_pdfs/`
 
-If a folder for a previous search already exists, create a new folder with a distinct descriptive search-session title rather than using versioned filenames.
+If a folder for a previous search on the same subject already exists, create a new folder with a distinct descriptive `<subject_of_study>` title rather than using versioned filenames.
 
 At the end of the task, clearly report the exact output folder and generated file paths to the user.
 
@@ -79,7 +80,6 @@ Use the custom literature tool as the primary search path:
 When calling `literature_search`:
 - Always construct `pubmed_query` using PubMed-specific syntax from the references below.
 - Use MeSH terms (`[mh]` / `[majr]`), title/abstract terms (`[tiab]`), publication types (`[pt]`), substance names (`[nm]`), date filters, and Boolean logic as appropriate.
-- Construct `semantic_scholar_query` separately as broader natural-language search terms when useful. Semantic Scholar is used automatically as supplementary search only when `SEMANTIC_SCHOLAR_API_KEY` is configured.
 - Do not pass a generic natural-language query as `pubmed_query` when a PubMed/MeSH query can be constructed.
 
 These extension tools are the preferred search path for this skill. Do not fall back to generic `WebFetch` / `WebSearch` first when one of these typed tools fits the task.
@@ -88,29 +88,15 @@ Read these references before constructing queries:
 - `references/pubmed_routine.md`
 - `references/pubmed_search_syntax.md`
 - `references/pubmed_common_queries.md`
-- `references/semanticscholar_routine.md`
 
 ### Step 4 — Screen and prioritise
 
-- Deduplicate across PubMed and Semantic Scholar sources.
-- Prioritise by relevance, recency, citation count, and study type.
+- Deduplicate PubMed results.
+- Prioritise by relevance, recency, and study type.
 - Default to deep reading of the top 20 papers unless the user asks otherwise.
 - For preclinical requests, keep studies with experimental target perturbation evidence.
 
-### Step 5 — Retrieve full text
-
-Use `fetch_fulltext` for top papers. Prefer it over ad-hoc `WebFetch` PDF retrieval because it applies the defined PMC → publisher OA → Sci-Hub chain.
-
-Access chain:
-1. PMC
-2. publisher open-access page
-3. Sci-Hub fallback
-
-Read:
-- `references/full-text-access-guide.md`
-- `references/scihub_routine.md`
-
-### Step 6 — Synthesis
+### Step 5 — Synthesis
 
 Always produce:
 1. a narrative synthesis with inline numbered citations
@@ -179,14 +165,13 @@ After reviewing the core paper set, optionally produce:
 
 ## Expected files
 
-Typical outputs must be placed in a dedicated search-session folder under `./results/`, for example `./results/literature_<descriptive_topic>/`:
+Typical outputs must be placed in a dedicated subject folder under `./results/literature_review/`, for example `./results/literature_review/<subject_of_study>/`:
 - `literature_report.md`
 - `paper_summary_table.csv`
 - `search_log.md`
-- `pdfs/`
 - optional `analysis_object.pkl` or other export artifacts when produced
 
-Do not write these outputs directly to `./results/` or reuse a previous search folder.
+Do not write these outputs directly to `./results/literature_review/` or to `./results/`, and do not reuse a previous subject folder.
 
 ## Companion references
 
@@ -194,10 +179,7 @@ Do not write these outputs directly to `./results/` or reuse a previous search f
 - `references/pubmed_routine.md`
 - `references/pubmed_search_syntax.md`
 - `references/pubmed_common_queries.md`
-- `references/semanticscholar_routine.md`
 - `references/preclinical-extraction-guide.md`
-- `references/full-text-access-guide.md`
-- `references/scihub_routine.md`
 
 ## Companion scripts
 
@@ -205,4 +187,3 @@ Do not write these outputs directly to `./results/` or reuse a previous search f
 - `scripts/synthesis.py`
 - `scripts/generate_table.py`
 - `scripts/export_all.py`
-- `scripts/scihub_pdf_resolver.py`

package/skills/literature/references/preclinical-extraction-guide.md

@@ -173,7 +173,7 @@ The `experiment_extraction.csv` file contains one row per paper with these colum
 ### 1. Abstract-only extraction
 The script only reads abstracts, not full text. Papers that describe experiments only in the methods/results sections (not the abstract) will be misclassified as "unclassified".
 
-**Mitigation:** Step 5 of the workflow (full-text enrichment) addresses this for top papers.
+**Mitigation:** No full-text enrichment step is currently available; papers whose experiments appear only in the methods/results sections may be misclassified as "unclassified".
 
 ### 2. Keyword sensitivity
 - **False positives:** A paper mentioning "mouse model" in the introduction (not as an experiment performed) may be classified as in_vivo.

package/skills/literature/scripts/generate_table.py

@@ -24,8 +24,6 @@ def _identifier(paper: Dict) -> str:
         return f"PMID:{paper['pmid']}"
     if paper.get("doi"):
         return paper["doi"]
-    if paper.get("s2_id"):
-        return paper["s2_id"]
     return "NA"
 
 
@@ -51,7 +49,7 @@ def build_table_rows(papers: List[Dict], experiments: List[Dict] | None = None,
             "#": idx,
             "PMID/DOI": _identifier(paper),
             "Authors (year)": _authors_year(paper),
-            "Key Message": _truncate(paper.get("tldr") or paper.get("title") or ""),
+            "Key Message": _truncate(paper.get("title") or ""),
             "Key Results": _truncate(paper.get("abstract") or exp.get("key_findings") or ""),
             "Key Methods": _truncate(
                 "; ".join(filter(None, [

package/skills/literature/scripts/synthesis.py

@@ -35,15 +35,16 @@ def classify_study_type(paper: Dict) -> str:
 
 def classify_evidence_quality(paper: Dict) -> str:
     study_type = classify_study_type(paper)
-    citation_count = int(paper.get("citation_count") or 0)
     if study_type in {"Systematic review / meta-analysis", "Randomized controlled trial"}:
         return "High"
     if study_type in {"Clinical study", "In vitro + in vivo"}:
         return "Moderate"
     if paper.get("is_preprint"):
         return "Preliminary (preprint)"
-    if study_type in {"In vivo", "In vitro"}:
-        return "Moderate" if citation_count >= 20 else "Low to moderate"
+    if study_type == "In vivo":
+        return "Moderate"
+    if study_type == "In vitro":
+        return "Preliminary"
     return "Preliminary"
 
 

package/src/index.ts

@@ -1,12 +1,8 @@
 import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
-import { registerFetchFulltextTool } from "./fulltext.ts";
 import { registerLiteratureSearchTool } from "./literature-search.ts";
 import { registerPubmedSearchTool } from "./pubmed.ts";
-import { registerSemanticScholarSearchTool } from "./semantic-scholar.ts";
 
 export default function literatureToolsExtension(pi: ExtensionAPI) {
   registerLiteratureSearchTool(pi);
   registerPubmedSearchTool(pi);
-  registerSemanticScholarSearchTool(pi);
-  registerFetchFulltextTool(pi);
 }

package/src/literature-search.ts

@@ -7,7 +7,6 @@ import {
   type LiteratureSearchDisplayEvent,
   type LiteratureSearchDisplaySearch,
 } from "./rendering.ts";
-import { searchSemanticScholar } from "./semantic-scholar.ts";
 import { formatPaperText, normalizeDoi, unique } from "./shared.ts";
 import { emitProgress, textResult, type TextToolUpdate } from "./tool-output.ts";
 import type { PaperRecord } from "./types.ts";
@@ -17,12 +16,6 @@ export const LITERATURE_SEARCH_PARAMS = Type.Object({
     description:
       "PubMed-ready query using PubMed syntax such as MeSH [mh], title/abstract [tiab], publication type [pt], substance [nm], and Boolean logic.",
   }),
-  semantic_scholar_query: Type.Optional(
-    Type.String({
-      description:
-        "Optional natural-language Semantic Scholar query for supplementary search. If omitted and Semantic Scholar is configured, a simplified query is derived from pubmed_query.",
-    }),
-  ),
   max_results: Type.Optional(
     Type.Number({ description: "Maximum results per provider (default 20)" }),
   ),
@@ -51,27 +44,11 @@ export type LiteratureSearchResult = {
   papers: PaperRecord[];
   providers: {
     pubmed: ProviderExecution;
-    semantic_scholar: ProviderExecution;
   };
   searches: LiteratureSearchDisplaySearch[];
   events: LiteratureSearchDisplayEvent[];
 };
 
-function firstYear(value?: string): number | undefined {
-  const match = value?.match(/^(\d{4})/);
-  return match?.[1] ? Number(match[1]) : undefined;
-}
-
-export function simplifyPubmedQueryForSemanticScholar(query: string): string {
-  const simplified = query
-    .replace(/\[[^\]]+\]/g, " ")
-    .replace(/\b(?:AND|OR|NOT)\b/gi, " ")
-    .replace(/[()"']/g, " ")
-    .replace(/\s+/g, " ")
-    .trim();
-  return simplified || query.trim();
-}
-
 function sourceList(paper: PaperRecord): string[] {
   return unique([
     ...(paper.sources ?? []),
@@ -92,7 +69,6 @@ function dedupeKeys(paper: PaperRecord): string[] {
   const keys = [
     doi ? `doi:${doi}` : undefined,
     paper.pmid ? `pmid:${paper.pmid}` : undefined,
-    paper.s2_id ? `s2:${paper.s2_id}` : undefined,
   ];
   const title = normalizedTitle(paper.title);
   if (title && paper.year) keys.push(`title-year:${title}:${paper.year}`);
@@ -106,7 +82,6 @@ function mergePapers(existing: PaperRecord, incoming: PaperRecord): PaperRecord
     ...existing,
     doi: normalizeDoi(existing.doi) ?? normalizeDoi(incoming.doi),
     pmid: existing.pmid ?? incoming.pmid,
-    s2_id: existing.s2_id ?? incoming.s2_id,
     title: existing.title !== "Untitled" ? existing.title : incoming.title,
     abstract: existing.abstract ?? incoming.abstract,
     authors: unique([...(existing.authors ?? []), ...(incoming.authors ?? [])]),
@@ -117,10 +92,6 @@ function mergePapers(existing: PaperRecord, incoming: PaperRecord): PaperRecord
       ...(incoming.publication_types ?? []),
     ]),
     mesh_terms: unique([...(existing.mesh_terms ?? []), ...(incoming.mesh_terms ?? [])]),
-    citation_count: existing.citation_count ?? incoming.citation_count,
-    tldr: existing.tldr ?? incoming.tldr,
-    open_access_pdf: existing.open_access_pdf ?? incoming.open_access_pdf,
-    external_ids: { ...(incoming.external_ids ?? {}), ...(existing.external_ids ?? {}) },
     source: sources.join(";"),
     sources,
   };
@@ -167,7 +138,7 @@ export async function searchLiterature(
     emitProgress(onUpdate, text, { events: [...events] });
   };
 
-  emitEvent("Starting literature search...");
+  emitEvent("Searching PubMed...");
 
   events.push({
     phase: "query_start",
@@ -175,7 +146,7 @@ export async function searchLiterature(
     query_index: 1,
     query: params.pubmed_query,
   });
-  emitEvent(`Searching PubMed q1: ${params.pubmed_query}`);
+  emitEvent("Searching PubMed...");
 
   const pubmed = await searchPubmed(
     {
@@ -204,98 +175,18 @@ export async function searchLiterature(
     query_index: 1,
     query: pubmed.query ?? params.pubmed_query,
     count: pubmed.count,
-    papers: pubmedDisplayPapers,
   });
-  emitEvent(`PubMed q1 found ${pubmed.count} candidate papers.`);
-
-  const semanticScholarApiKey = process.env.SEMANTIC_SCHOLAR_API_KEY?.trim();
-  let semanticScholar: ProviderExecution = {
-    searched: false,
-    reason: "SEMANTIC_SCHOLAR_API_KEY not configured",
-  };
-  let semanticScholarPapers: PaperRecord[] = [];
-
-  if (semanticScholarApiKey) {
-    const semanticScholarQuery =
-      params.semantic_scholar_query?.trim() ||
-      simplifyPubmedQueryForSemanticScholar(params.pubmed_query);
-
-    events.push({
-      phase: "query_start",
-      provider: "semantic_scholar",
-      query_index: 1,
-      query: semanticScholarQuery,
-    });
-    emitEvent(`Searching Semantic Scholar q1: ${semanticScholarQuery}`);
-
-    try {
-      const semanticScholarResult = await searchSemanticScholar(
-        {
-          query: semanticScholarQuery,
-          max_results: Math.min(100, maxResults),
-          year_from: firstYear(params.date_from),
-          year_to: firstYear(params.date_to),
-        },
-        signal,
-        undefined,
-      );
-      semanticScholarPapers = semanticScholarResult.papers;
-      const semanticScholarDisplayPapers = compactPapersForDisplay(
-        semanticScholarResult.papers,
-      );
-      searches.push({
-        provider: "semantic_scholar",
-        query_index: 1,
-        query: semanticScholarQuery,
-        count: semanticScholarResult.count,
-        papers: semanticScholarDisplayPapers,
-      });
-      events.push({
-        phase: "query_results",
-        provider: "semantic_scholar",
-        query_index: 1,
-        query: semanticScholarQuery,
-        count: semanticScholarResult.count,
-        papers: semanticScholarDisplayPapers,
-      });
-      emitEvent(
-        `Semantic Scholar q1 found ${semanticScholarResult.count} candidate papers.`,
-      );
-      semanticScholar = {
-        searched: true,
-        count: semanticScholarResult.count,
-        query: semanticScholarQuery,
-      };
-    } catch (err) {
-      const message = err instanceof Error ? err.message : String(err);
-      events.push({
-        phase: "query_error",
-        provider: "semantic_scholar",
-        query_index: 1,
-        query: semanticScholarQuery,
-        error: message,
-      });
-      semanticScholar = {
-        searched: false,
-        reason: `Semantic Scholar search failed: ${message}`,
-      };
-      emitEvent(`Semantic Scholar q1 failed: ${message}`);
-    }
-  }
+  emitEvent(`Found ${pubmed.count} PubMed ${pubmed.count === 1 ? "paper" : "papers"}.`);
 
   events.push({ phase: "dedupe" });
-  emitEvent("Deduplicating literature results...");
+  emitEvent("Preparing results...");
 
-  const papers = dedupeLiteraturePapers([
-    ...pubmed.papers,
-    ...semanticScholarPapers,
-  ]);
+  const papers = dedupeLiteraturePapers(pubmed.papers);
   events.push({
     phase: "complete",
     count: papers.length,
-    papers: compactPapersForDisplay(papers),
   });
-  emitEvent(`Literature search complete: ${papers.length} merged papers.`);
+  emitEvent(`Literature search complete: ${papers.length} PubMed ${papers.length === 1 ? "paper" : "papers"}.`);
 
   return {
     count: papers.length,
@@ -307,7 +198,6 @@ export async function searchLiterature(
         query: pubmed.query ?? params.pubmed_query,
         total: pubmed.total,
       },
-      semantic_scholar: semanticScholar,
     },
     searches,
     events,
@@ -319,7 +209,7 @@ export function createLiteratureSearchTool() {
     name: "literature_search",
     label: "Literature Search",
     description:
-      "Run the literature workflow search: PubMed is always searched first with a PubMed-ready query; Semantic Scholar is searched as supplementary metadata when SEMANTIC_SCHOLAR_API_KEY is configured.",
+      "Run the literature workflow search against PubMed using a PubMed-ready query (MeSH [mh], title/abstract [tiab], publication type [pt], substance [nm], and Boolean logic).",
     parameters: LITERATURE_SEARCH_PARAMS,
     async execute(
       _toolCallId: string,