@wentorai/research-plugins 1.1.0 → 1.2.1

package/skills/literature/fulltext/pmc-ftp-bulk-download/SKILL.md

@@ -0,0 +1,182 @@
+---
+name: pmc-ftp-bulk-download
+description: "Bulk download PMC Open Access articles via FTP for large-scale mining"
+metadata:
+  openclaw:
+    emoji: "📦"
+    category: "literature"
+    subcategory: "fulltext"
+    keywords: ["pmc", "bulk download", "ftp", "text mining", "open access", "pubmed central"]
+    source: "https://www.ncbi.nlm.nih.gov/pmc/tools/ftp/"
+---
+
+# PMC FTP Bulk Download
+
+## Overview
+
+The PMC FTP Service provides bulk download access to millions of full-text articles from PubMed Central's Open Access Subset. Unlike the single-article APIs (E-utilities, BioC), the FTP service is designed for large-scale corpus construction — downloading entire collections for text mining, NLP training, systematic reviews, and bibliometric analysis. Free, no authentication required.
+
+**Note**: PMC is migrating to AWS-based Cloud Service in August 2026. FTP paths may change; check official docs for updates.
+
+## FTP Access Points
+
+### Connection
+
+```bash
+# FTP (classic)
+ftp ftp.ncbi.nlm.nih.gov
+# Navigate to: /pub/pmc
+
+# HTTPS alternative (recommended)
+# Base: https://ftp.ncbi.nlm.nih.gov/pub/pmc/
+```
+
+### Available Datasets
+
+| Dataset | Path | Content | Format |
+|---------|------|---------|--------|
+| **OA Commercial** | `/pub/pmc/oa_comm/` | CC BY/CC0 articles (commercial use OK) | .tar.gz packages |
+| **OA Non-Commercial** | `/pub/pmc/oa_noncomm/` | CC BY-NC articles | .tar.gz packages |
+| **OA Other** | `/pub/pmc/oa_other/` | Other open licenses | .tar.gz packages |
+| **Author Manuscripts** | `/pub/pmc/manuscript/` | NIH-funded manuscripts | .tar.gz packages |
+| **Historical OCR** | `/pub/pmc/historical_ocr/` | Pre-digital scanned articles | .tar.gz |
+| **File lists** | `/pub/pmc/oa_file_list.csv` | Index of all OA articles | CSV |
+
+### File List Index
+
+Download the master index to plan your downloads:
+
+```bash
+# Download the OA file list (CSV, ~200MB)
+wget https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_file_list.csv
+
+# CSV columns:
+# File, Article Citation, AccessionID, LastUpdated, PMID, License
+```
+
+## Download Strategies
+
+### Strategy 1: Download Specific Articles
+
+```python
+import requests
+import tarfile
+import io
+import csv
+
+def download_article_package(pmcid: str, base_url: str = "https://ftp.ncbi.nlm.nih.gov/pub/pmc"):
+    """Download and extract a specific PMC article package."""
+    # First, look up the file path from the file list
+    # (In practice, you'd load this once and index by PMCID)
+    file_list_url = f"{base_url}/oa_file_list.csv"
+    # ... lookup pmcid in file list to get path ...
+
+    # Download the tar.gz package
+    resp = requests.get(f"{base_url}/{file_path}", stream=True)
+    resp.raise_for_status()
+
+    # Extract
+    with tarfile.open(fileobj=io.BytesIO(resp.content), mode="r:gz") as tar:
+        tar.extractall(path=f"./articles/{pmcid}")
+    print(f"Extracted {pmcid}")
+```
+
+### Strategy 2: Bulk Download by License
+
+```bash
+#!/bin/bash
+# Download all commercial-use articles (CC BY / CC0)
+# WARNING: This is ~100GB+ compressed
+
+mkdir -p pmc_corpus/commercial
+cd pmc_corpus/commercial
+
+# Download the baseline (all current articles)
+wget -r -np -nH --cut-dirs=3 \
+  https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_comm/xml/
+
+# Incremental updates (run periodically)
+wget -r -np -nH --cut-dirs=3 -N \
+  https://ftp.ncbi.nlm.nih.gov/pub/pmc/oa_comm/xml/
+```
+
+### Strategy 3: Filtered Download via File List
+
+```python
+import csv
+import requests
+from pathlib import Path
+
+def download_filtered_corpus(file_list_path: str, output_dir: str,
+                              license_filter: str = "CC BY",
+                              max_articles: int = 1000):
+    """Download articles matching a license filter."""
+    output = Path(output_dir)
+    output.mkdir(parents=True, exist_ok=True)
+    base = "https://ftp.ncbi.nlm.nih.gov/pub/pmc"
+    downloaded = 0
+
+    with open(file_list_path) as f:
+        reader = csv.DictReader(f)
+        for row in reader:
+            if license_filter and license_filter not in row.get("License", ""):
+                continue
+            if downloaded >= max_articles:
+                break
+
+            file_path = row["File"]
+            url = f"{base}/{file_path}"
+            local_path = output / Path(file_path).name
+
+            if local_path.exists():
+                continue
+
+            resp = requests.get(url, stream=True, timeout=60)
+            if resp.status_code == 200:
+                local_path.write_bytes(resp.content)
+                downloaded += 1
+                if downloaded % 100 == 0:
+                    print(f"Downloaded {downloaded} articles...")
+
+    print(f"Total downloaded: {downloaded}")
+```
+
+## PMC ID Cross-Referencing
+
+Convert between different article identifiers:
+
+```bash
+# PMID → PMCID → DOI conversion
+curl "https://www.ncbi.nlm.nih.gov/pmc/utils/idconv/v1.0/?ids=29346600&format=json"
+
+# Batch conversion (up to 200 IDs)
+curl "https://www.ncbi.nlm.nih.gov/pmc/utils/idconv/v1.0/?ids=29346600,30266829,31048553&format=json"
+```
+
+## Package Contents
+
+Each article package (.tar.gz) typically contains:
+
+```
+PMC1234567/
+├── PMC1234567.xml       # Full text in JATS XML
+├── PMC1234567.pdf       # PDF (if available)
+├── figure1.jpg          # Figures
+├── figure2.jpg
+├── table1.html          # Tables (sometimes)
+└── supplement1.pdf      # Supplementary materials
+```
+
+## Best Practices
+
+- **Start with the file list**: Download `oa_file_list.csv` first and filter locally
+- **Respect rate limits**: Space requests 0.3s apart for individual downloads
+- **Use incremental updates**: After initial download, use `-N` flag to only get new/updated files
+- **Check licenses**: OA Commercial (CC BY) allows any use; Non-Commercial restricts commercial applications
+- **Storage planning**: Full OA Subset is ~500GB+ uncompressed
+
+## References
+
+- [PMC FTP Documentation](https://www.ncbi.nlm.nih.gov/pmc/tools/ftp/)
+- [PMC Open Access Subset](https://www.ncbi.nlm.nih.gov/pmc/tools/openftlist/)
+- [PMC ID Converter API](https://www.ncbi.nlm.nih.gov/pmc/tools/id-converter-api/)

package/skills/literature/fulltext/zotero-ai-butler-guide/SKILL.md

@@ -0,0 +1,166 @@
+---
+name: zotero-ai-butler-guide
+description: "AI-powered paper summarization plugin for Zotero"
+metadata:
+  openclaw:
+    emoji: "🤵"
+    category: "literature"
+    subcategory: "fulltext"
+    keywords: ["Zotero", "AI summary", "paper summarization", "LLM", "abstract generation", "reading assistant"]
+    source: "https://github.com/steven-jianhao-li/zotero-AI-Butler"
+---
+
+# Zotero AI Butler Guide
+
+## Overview
+
+Zotero AI Butler is a Zotero plugin that uses LLMs to summarize, analyze, and annotate academic papers directly within Zotero. It can generate structured summaries, extract key findings, compare papers, and answer questions about documents — all without leaving the reference manager. Supports multiple LLM backends (OpenAI, Claude, local models).
+
+## Installation
+
+```bash
+# Download .xpi from GitHub releases
+# Zotero 7: Tools → Add-ons → Install Add-on From File
+```
+
+## Configuration
+
+```markdown
+### LLM Backend Setup (Preferences → AI Butler)
+
+**Option 1: OpenAI**
+- Provider: OpenAI
+- Model: gpt-4o
+- Set environment variable for credentials
+
+**Option 2: Anthropic**
+- Provider: Anthropic
+- Model: claude-sonnet-4-20250514
+
+**Option 3: Local (Ollama)**
+- Provider: Ollama
+- Endpoint: http://localhost:11434
+- Model: llama3.1
+
+**Option 4: Custom API**
+- Provider: Custom
+- Endpoint: your-api-url
+- Compatible with OpenAI API format
+```
+
+## Features
+
+### Paper Summarization
+
+```markdown
+### Usage
+1. Select paper in Zotero
+2. Right-click → AI Butler → Summarize
+3. Summary added as Zotero note
+
+### Summary Templates
+- **Quick Summary** (1 paragraph): Core contribution + method + result
+- **Structured Summary**: Background / Method / Results / Limitations
+- **Executive Brief**: Who should read this and why
+- **Technical Deep-Dive**: Detailed methodology and math
+```
+
+### Key Finding Extraction
+
+```markdown
+### Extract structured information:
+- **Research question**: What problem does this paper address?
+- **Methodology**: What approach do the authors use?
+- **Key results**: What are the main findings?
+- **Contributions**: What is novel about this work?
+- **Limitations**: What are the acknowledged limitations?
+- **Future work**: What directions do the authors suggest?
+```
+
+### Paper Comparison
+
+```markdown
+### Compare multiple papers:
+1. Select 2+ papers in Zotero
+2. Right-click → AI Butler → Compare Papers
+3. Generates comparison table:
+   - Shared and unique contributions
+   - Methodological differences
+   - Performance comparison (if applicable)
+   - Complementary insights
+```
+
+### Q&A Mode
+
+```markdown
+### Ask questions about papers:
+1. Open paper in Zotero reader
+2. AI Butler sidebar → Ask a question
+3. Answers grounded in paper content with page references
+
+Example questions:
+- "What loss function do they use?"
+- "How does this compare to prior work?"
+- "What are the hyperparameters?"
+- "Explain equation 3 in simpler terms"
+```
+
+## Batch Processing
+
+```markdown
+### Summarize multiple papers:
+1. Select papers (or entire collection)
+2. Right-click → AI Butler → Batch Summarize
+3. Progress bar shows completion
+4. Each paper gets a summary note attached
+
+### Reading List Generation:
+1. Select collection
+2. AI Butler → Generate Reading Order
+3. Suggests optimal reading sequence based on:
+   - Citation relationships
+   - Conceptual dependencies
+   - Publication chronology
+```
+
+## Custom Prompts
+
+```markdown
+### Create custom analysis prompts:
+# In AI Butler preferences → Custom Prompts
+
+Prompt: "Systematic Review Extraction"
+Template: |
+  Extract the following from this paper:
+  1. Study design (RCT, cohort, etc.)
+  2. Sample size
+  3. Primary outcome
+  4. Effect size with CI
+  5. Risk of bias indicators
+  Format as structured JSON.
+```
+
+## Integration with Zotero Workflow
+
+```markdown
+### Combined Plugin Workflow
+1. **Zotero Connector** → Import paper
+2. **Zotero Sci-Hub** → Fetch PDF
+3. **AI Butler** → Generate summary note
+4. **Zotero Actions Tags** → Auto-tag based on summary
+5. **Notero** → Sync to Notion with summary
+6. **Better BibTeX** → Export citations for writing
+```
+
+## Use Cases
+
+1. **Rapid screening**: Quick summaries for literature triage
+2. **Paper comprehension**: Ask clarifying questions
+3. **Comparison studies**: Side-by-side paper analysis
+4. **Data extraction**: Structured information for systematic reviews
+5. **Reading preparation**: Generate briefings before journal club
+
+## References
+
+- [Zotero AI Butler GitHub](https://github.com/steven-jianhao-li/zotero-AI-Butler)
+- [Zotero Plugin Development](https://www.zotero.org/support/dev/client_coding/plugin_development)

package/skills/literature/fulltext/zotero-scihub-guide/SKILL.md

@@ -0,0 +1,168 @@
+---
+name: zotero-scihub-guide
+description: "Zotero plugin for automatic PDF retrieval from Sci-Hub"
+metadata:
+  openclaw:
+    emoji: "🔓"
+    category: "literature"
+    subcategory: "fulltext"
+    keywords: ["Zotero", "Sci-Hub", "PDF download", "open access", "full text", "paper retrieval"]
+    source: "https://github.com/ethanwillis/zotero-scihub"
+---
+
+# Zotero Sci-Hub Guide
+
+## Overview
+
+Zotero Sci-Hub is a Zotero plugin that automatically fetches PDFs from Sci-Hub when papers cannot be found through standard open-access channels. It integrates seamlessly into Zotero's existing PDF retrieval workflow — when Zotero's built-in retriever fails, the plugin automatically attempts Sci-Hub as a fallback. Useful for researchers at institutions with limited journal subscriptions.
+
+## Installation
+
+```bash
+# Download the .xpi file from GitHub releases
+# In Zotero 7: Tools → Add-ons → Install Add-on From File
+
+# Manual installation
+# 1. Go to https://github.com/ethanwillis/zotero-scihub/releases
+# 2. Download zotero-scihub-*.xpi
+# 3. In Zotero: Tools → Add-ons → gear icon → Install from file
+```
+
+## Configuration
+
+```
+# In Zotero: Edit → Preferences → Zotero Sci-Hub
+
+# Settings:
+# 1. Sci-Hub URL: Set current working mirror
+#    - The plugin ships with default URLs
+#    - Update if mirrors change
+
+# 2. Automatic mode:
+#    - ON: Try Sci-Hub automatically after Zotero fails
+#    - OFF: Only fetch via right-click menu
+
+# 3. DOI sources: Where to look for DOIs
+#    - Item DOI field
+#    - Item URL field
+#    - Item Extra field
+```
+
+## Usage Workflow
+
+```markdown
+### Automatic PDF Retrieval
+1. Add item to Zotero (via browser connector, DOI, or manual)
+2. Zotero tries built-in PDF retrieval (Open Access, institutional)
+3. If no PDF found → plugin automatically queries Sci-Hub
+4. PDF attached to Zotero item
+
+### Manual Retrieval
+1. Right-click item(s) in Zotero
+2. Select "Fetch PDF from Sci-Hub"
+3. Works for single items or batch selection
+
+### Bulk Retrieval
+1. Select multiple items (Ctrl+A for all)
+2. Right-click → "Fetch PDF from Sci-Hub"
+3. Plugin processes items sequentially with rate limiting
+```
+
+## Integration with Other Plugins
+
+```markdown
+### Recommended Plugin Stack
+1. **Zotero Connector** — Browser extension for importing items
+2. **Zotero Sci-Hub** — PDF fallback retrieval
+3. **ZotFile/ZotMoov** — Organize downloaded PDFs
+4. **Zotero Better BibTeX** — Citation key management
+5. **Zotero PDF Translate** — Translate retrieved papers
+
+### Workflow
+Import item → Auto-fetch PDF → Organize files → Read & annotate
+```
+
+## Troubleshooting
+
+```markdown
+### Common Issues
+
+**PDF not found:**
+- Check if DOI is present in item metadata
+- Try updating the Sci-Hub mirror URL
+- Some very recent papers may not be available yet
+
+**Connection errors:**
+- Current mirror may be down; try alternate URL
+- Check network/proxy settings
+- Some institutions block Sci-Hub domains
+
+**Duplicate PDFs:**
+- Disable automatic mode if using other PDF fetchers
+- Check Zotero's duplicate detection settings
+```
+
+## Programmatic Alternative
+
+```python
+# For building custom PDF retrieval pipelines
+import requests
+
+def fetch_paper_by_doi(doi, output_path):
+    """Attempt to fetch paper PDF via DOI resolution."""
+    # Try Unpaywall first (legal open access)
+    unpaywall_url = (
+        f"https://api.unpaywall.org/v2/{doi}"
+        f"?email=your@email.com"
+    )
+    resp = requests.get(unpaywall_url)
+    if resp.ok:
+        data = resp.json()
+        if data.get("is_oa") and data.get("best_oa_location"):
+            pdf_url = data["best_oa_location"].get("url_for_pdf")
+            if pdf_url:
+                pdf = requests.get(pdf_url)
+                with open(output_path, "wb") as f:
+                    f.write(pdf.content)
+                return True
+
+    # Try CORE API
+    core_url = f"https://api.core.ac.uk/v3/search/works?q=doi:{doi}"
+    resp = requests.get(core_url)
+    if resp.ok:
+        results = resp.json().get("results", [])
+        if results and results[0].get("downloadUrl"):
+            pdf = requests.get(results[0]["downloadUrl"])
+            with open(output_path, "wb") as f:
+                f.write(pdf.content)
+            return True
+
+    return False
+```
+
+## Legal Considerations
+
+```markdown
+### Open Access Alternatives
+Before using Sci-Hub, check these legal sources:
+1. **Unpaywall** — Browser extension for legal OA versions
+2. **CORE** — Aggregator of OA research papers
+3. **PubMed Central** — Free biomedical literature archive
+4. **arXiv/bioRxiv** — Preprint servers
+5. **Author websites** — Many post preprints freely
+6. **Interlibrary Loan** — Request through your library
+7. **Email the author** — Most researchers share on request
+```
+
+## Use Cases
+
+1. **PDF retrieval**: Automatic paper downloading for Zotero
+2. **Literature collection**: Build reading libraries efficiently
+3. **Systematic reviews**: Bulk-fetch papers for review pipelines
+4. **Research onboarding**: Quickly gather papers for new topics
+
+## References
+
+- [Zotero Sci-Hub GitHub](https://github.com/ethanwillis/zotero-scihub)
+- [Unpaywall](https://unpaywall.org/) — Legal OA alternative
+- [CORE](https://core.ac.uk/) — OA aggregator

package/skills/literature/metadata/bibliometrix-guide/SKILL.md

@@ -0,0 +1,164 @@
+---
+name: bibliometrix-guide
+description: "Perform science mapping and bibliometric analysis with R bibliometrix"
+metadata:
+  openclaw:
+    emoji: "📉"
+    category: "literature"
+    subcategory: "metadata"
+    keywords: ["bibliometrix", "bibliometrics", "science mapping", "R", "citation analysis", "research trends"]
+    source: "https://github.com/massimoaria/bibliometrix"
+---
+
+# Bibliometrix Guide
+
+## Overview
+
+Bibliometrix is an R package for comprehensive science mapping and bibliometric analysis. It imports data from Scopus, Web of Science, PubMed, and other databases, then performs co-citation analysis, keyword co-occurrence mapping, collaboration networks, thematic evolution tracking, and more. Includes Biblioshiny — a Shiny-based web interface for no-code analysis.
+
+## Installation
+
+```r
+install.packages("bibliometrix")
+
+# Or development version
+devtools::install_github("massimoaria/bibliometrix")
+```
+
+## Quick Start
+
+### Import Data
+
+```r
+library(bibliometrix)
+
+# From Scopus CSV export
+M <- convert2df("scopus_export.csv", dbsource = "scopus", format = "csv")
+
+# From Web of Science
+M <- convert2df("wos_export.txt", dbsource = "wos", format = "plaintext")
+
+# From PubMed
+M <- convert2df("pubmed_export.txt", dbsource = "pubmed", format = "pubmed")
+
+# From multiple files
+file_list <- c("data1.csv", "data2.csv")
+M <- convert2df(file_list, dbsource = "scopus", format = "csv")
+```
+
+### Descriptive Analysis
+
+```r
+# Basic bibliometric summary
+results <- biblioAnalysis(M)
+summary(results, k = 10)  # Top 10 in each category
+
+# Key metrics produced:
+# - Publication trends over time
+# - Most productive authors
+# - Most cited papers
+# - Top journals/sources
+# - Country/affiliation rankings
+# - Keyword frequency
+```
+
+### Citation Analysis
+
+```r
+# Most cited documents
+CR <- citations(M, field = "article", sep = ";")
+head(CR$Cited, 20)
+
+# Most cited first authors
+CR_auth <- citations(M, field = "author", sep = ";")
+
+# Local citations (within the dataset)
+LC <- localCitations(M)
+head(LC$Papers, 10)
+```
+
+### Network Analysis
+
+```r
+# Co-citation network
+NetMatrix <- biblioNetwork(M, analysis = "co-citation",
+                           network = "references", sep = ";")
+net <- networkPlot(NetMatrix, n = 30, type = "fruchterman",
+                   Title = "Co-citation Network")
+
+# Author collaboration network
+NetMatrix <- biblioNetwork(M, analysis = "collaboration",
+                           network = "authors", sep = ";")
+net <- networkPlot(NetMatrix, n = 50, type = "kamada",
+                   Title = "Collaboration Network")
+
+# Keyword co-occurrence
+NetMatrix <- biblioNetwork(M, analysis = "co-occurrences",
+                           network = "keywords", sep = ";")
+net <- networkPlot(NetMatrix, n = 40, type = "fruchterman",
+                   Title = "Keyword Co-occurrence")
+```
+
+### Thematic Analysis
+
+```r
+# Thematic map (strategic diagram)
+Map <- thematicMap(M, field = "DE", n = 250, minfreq = 5)
+plot(Map$map)
+
+# Quadrants:
+# Motor themes (high centrality, high density)
+# Basic themes (high centrality, low density)
+# Niche themes (low centrality, high density)
+# Emerging/declining themes (low centrality, low density)
+
+# Thematic evolution over time periods
+nexus <- thematicEvolution(M,
+    field = "DE",
+    years = c(2015, 2019, 2023),
+    n = 100, minFreq = 3)
+plotThematicEvolution(nexus$Nodes, nexus$Edges)
+```
+
+### Biblioshiny (Web Interface)
+
+```r
+# Launch interactive web dashboard
+biblioshiny()
+
+# Opens browser with GUI for:
+# - Data import from multiple sources
+# - Descriptive analysis
+# - Network visualization
+# - Thematic mapping
+# - All plots exportable
+```
+
+## Supported Data Sources
+
+| Source | Format | Import function |
+|--------|--------|----------------|
+| Scopus | CSV/BibTeX | `convert2df(..., dbsource="scopus")` |
+| Web of Science | Plain text/BibTeX | `convert2df(..., dbsource="wos")` |
+| PubMed | PubMed format | `convert2df(..., dbsource="pubmed")` |
+| Dimensions | CSV | `convert2df(..., dbsource="dimensions")` |
+| Cochrane | Plain text | `convert2df(..., dbsource="cochrane")` |
+| OpenAlex | JSON | Via API integration |
+
+## Key Analysis Types
+
+| Analysis | Function | Output |
+|----------|----------|--------|
+| Descriptive | `biblioAnalysis()` | Summary statistics |
+| Co-citation | `biblioNetwork(analysis="co-citation")` | Citation clusters |
+| Collaboration | `biblioNetwork(analysis="collaboration")` | Author networks |
+| Co-occurrence | `biblioNetwork(analysis="co-occurrences")` | Keyword maps |
+| Thematic map | `thematicMap()` | Strategic quadrant diagram |
+| Trend analysis | `fieldByYear()` | Topic evolution |
+| Country collab | `metaTagExtraction() + biblioNetwork()` | Geo collaboration |
+
+## References
+
+- [Bibliometrix](https://www.bibliometrix.org/)
+- [Bibliometrix GitHub](https://github.com/massimoaria/bibliometrix)
+- Aria, M. & Cuccurullo, C. (2017). "bibliometrix: An R-tool for comprehensive science mapping analysis." *Journal of Informetrics* 11(4): 959-975.